November patch updates to Windows Server 2008r2, 2012, 2012r2, 2016, and 2019 will cause a memory leak in the LSASS on Domain Controllers, causing them to freeze and have to be hard restarted. This is after the March patch LSASS disaster causing Domain Controller reboots. There is a temporary workaround in the BleepingComputer article link, with Microsoft promising a "fix" in the near future. We're all simply Microsoft's QA test pool now. Where's my eye roll emoji when I need it???
There are times when you need to remove a drive or set of drives from an IBM Storwize v3700, v5000, or v7000 series storage system. Removal and addition of disks, especially if you're replacing drives with refurbished/used disks, is a command-line task. You connect to the Storwize vX000 array using Putty or a similar SSH client.
All commands for reference purposes are in the link below. Available commands will vary by firmware version.
Do this to fail-out disks before removal:
svctask chdrive -allowdegraded -use failed XX (where XX is the drive unit number determined in the GUI)
Do this to clear failed-out disks from the GUI:
svctask chdrive -use failed XX (where XX is the drive ID unit number)
svctask chdrive -use unused XX
Do this to mark added disks as candidate disk:
svctask chdrive -use candidate XX (where XX is drive ID unit number)
Do this to clear dump space after failed firmware upgrade, before re-attempting the upgrade:
cleardumps -prefix /dumps
cleardumps -prefix /home/admin/upgrade
All versions of Veeam include the Veeam.Backup.Validator.exe command-line function. This helpful tool can be used to verify the integrity of your backup files on a daily basis, without the need to go through the hassles of setting up SureBackup. Backup files can become corrupted for a variety of reasons. The file can be damaged during transfer over the network or due to hardware failures on the backup storage. Using the Veeam Backup Validator you can quickly verify the integrity of any backup file without having to extract the VM data from the archive.
Veeam Backup Validator is a CRC-check utility that uses the checksum algorithm to validate file integrity. When Veeam Backup and Replication creates a backup of a VM, it calculates a checksum for every data block in the backup file and attaches these checksums to the data blocks. Veeam Backup Validator recalculates checksums for the data blocks and compares them against the initial checksum values. If the results match, the backup file is viable. This works similarly to the backup file integrity check performed by SureBackup.
The Veeam.Backup.Validator.exe file is located by default in the installation folder of Veeam Backup and Replication: %ProgramFiles%\Veeam\Backup and Replication\Backup\Veeam.Backup.Validator.exe. The utility must be run from a command prompt on the backup server. If you attempt to run it from a remote Veeam management conole install, the utility will fail. To run the validator utility an account with administraive rights is required. You can display all of the command switch options by typing Veeam.Backup.Validator.exe /?.
The syntax of the command is as follows: Veeam.Backup.Validator.exe /backup:backupname|backupid [/vmname:vmname] [/point:pointid] [/date:pointdate] [/time:pointtime] [/silence] [/skip] [/report:reportpath [/format:xml|html]] The [ ] square brackets indicate options, do not include the brackets in the command.
There are several ways to automate this process. One way is to create a powershell script or a batch file and call that file from a Windows scheduled task. Here's an example of a simple batch file:
set VBR01=Production Backups
c:\"program files"\veeam\"backup and replication"\backup\veeam.backup.validator.exe /backup:"%VBR01%" /report:"c:\vbrreports\%VBR01%_%TODAYSDATE%.html" /format:html
rem c:\"program files"\veeam\"backup and replication"\backup\veeam.backup.validator.exe /backup:"%VBR02%" /report:"c:\vbrreports\%VBR02%_%TODAYSDATE%.html" /format:html
c:\"program files"\veeam\"backup and replication"\backup\veeam.backup.validator.exe /backup:"%VBR03%" /report:"c:\vbrreports\%VBR03%_%TODAYSDATE%.html" /format:html
@echo "Validations Complete"
This batch file will run the validation against each daily backup job and output the results to an HTML formatted document.
Here's a link to the Veeam Help Center User Guide that gives more details regarding each of the command-line parameters and how they can be used.
Windows File Server Resource Manager (FSRM) has File Management tasks within the FSRM console. File Management tasks can be run out of the box on a Windows Server 2008 R2 or newer system, to either expire classified files that meet a certain criteria, by moving these files to a designated folder location, or to perform a custom task. This can be a handy tool to automatically move files that have not been accessed in an extended period of time. Or, in the case of sensitive data, such as files that might contain passwords, this tool can be used to create a custom script to move the classified files to a designated, secured folder, and leave a link or note in the original location to instruct any users on how to regain access to that file.
Rootusers has a short tutorial on how to set up a file movement task via FSRM:
If you have installed a Portable or Linux OS on a thumb drive or SD Card, Windows typically won't be able to reformat it for other use. To work around the limitations of File Explorer you can call DiskPart into service to help solve this problem.
Insert the USB Thumb Drive or SD Card
Run CMD.exe as Administrator
DISKPART> select disk 1
Disk 1 is now the selected disk
DISKPART> List Disk (to verify selection (marked with asterisk to left of row)
*VERY Important Note: in my case since I only have a single internal hard disk in my laptop, the USB drive shows as disk 1, an 8GB stick. If you have more than one disk already installed/mounted on your PC, then modify the select statement accordingly to point to your target USB device. If you select the wrong disk you could wipe out all of the data on your PC.
DiskPart succeeded in cleaning the disk.
DISKPART>create partition primary
DiskPart succeeded in creating the specified partition.
DiskPart marked the current partition active.
DISKPART>format fs=fat32 quick
100 percent completed.
DiskPart successfully formatted the volume.
DiskPart successfully assigned the drive letter or mount point.
C:\> exit (to exit CMD window)
This was a LINKEDIN post I made on Dec 20, 2021. I'm reposting here for ease of reference.
There are two detailed articles from VMware outlining the steps necessary within the VMware vCenter appliance for modifying the files to insulate your vCenter appliance from the Apache log4j2 vulnerability. However, VMware leaves out a few pre-requisites and interim steps that you might need to know about to make this task easier.
First, let's start with the prereqs. You'll need to download the python scripts attached to the two VMware articles that outline the remediation steps (links provided below). Then, you're going to need Putty or some other SSH client to be able to log in to the vCenter appliance and run these scripts. Thirdly, you'll need WinSCP or a similar file transfer client to move the files from your PC to the vCenter appliance.
Once you have the three python script files downloaded (vmsa-2021-0028-kb87081.py, remove_log4j_class.py and vc_log4j_mitigator.py), launch your web browser and navigate to the admin interface for your vCenter appliance (https://vcenterdnsname:5480), and log in as root. This is not the root user for your ESX host, it's the root user for the vCenter appliance. If you can't log in, your root password may have expired since for some idiotic reason VMware thought it was a good idea to set the default for root's password to automatically expire in 90 days, after which you're permanently locked out and there is no option to add another root-equivalent user. Anyway, I've provided a link to an article below on how to reset the root password of a vCenter appliance, should that be necessary. Once you've logged in as root, navigate on the lefthand menu to the Access submenu, then go to the upper right side and click EDIT, to enable the BASH SHELL. Set the timeout to 30 minutes or longer. Log out of the admin interface.
Next, navigate to the main menu of the vCenter server (https://vcenterdnsname) and log in using either email@example.com or an Active Directory admin user authorized for vCenter. Find your vCenter appliance in the VMs list, right-click on the VM, go to Snapshots > Take a Snapshot > OK and then wait for that to finish (the user interface may pause for a moment while this happens). Verify that the snapshot has been completed, and then log out.
Now, we're going to transfer the python scripts to the vCenter server. I'll use the WinSCP example here. Since the default shell environment for vCenter is the appliancesh shell, and not bash shell, you'll have to do one of two things. Either SSH to the vCenter appliance and change the default to BASH shell to be able to use secure copy (SCP), or simply use SFTP which can be accessed when the appliancesh is the default shell environment.
In WinSCP new session, select SFTP as the file protocol, type in the host IP address or DNS name of the vCenter appliance, leave port as 22, and the username will be root and type in your root password, then click advanced. See the screenshot above for the protocol options to replace "Default" in the SFTP server block, with shell /usr/libexec/sftp-server, then click OK to close that windows and click LOGIN. Navigate in the righthand pane of the WinSCP client to the ./tmp folder and copy the two python scripts to this folder. You can then end your WinSCP session.
Launch your putty client to connect to the vCenter server via SSH, port 22. If you need to document session input and output for audit reasons, make sure to enable logging in the putty client. Once connected, log in as root. You'll be presented a Command> prompt. Type in shell. You'll then be given a root@vcsaXX [ ~ ]# prompt (with the version of the vCenter server replacing the XX. Type in cd /tmp and press enter. You can then type ls -l *.py to verify that the three python scripts are in this directory.
Now, let's pause here to consider the potential impact of the next two actions. When you run the python scripts several services are STOPPED on the vCenter appliance. This could impact any service that depends on vCenter, such as VMware replication, VMware Backup, Zerto Replication, Veeam Replication, Veeam CDP, Veeam Backup, and other such programs as well as anyone attempting to use the GUI interface of vCenter. Before you run the python scripts you'll need to notify users of the outage and potentially pause other programs to reduce the possibility of an interruption of services for replication or backups. The two article links below outline the services that will be stopped by the python scripts. Review those to determine any potential impact before running the two scripts.
Now that you've considered the consequences of these next steps and taken actions to reduce any potential impact, you can run the first script by typing: python vmsa-2021-0028-kb87081.py from the root@vcsaXX [ /tmp ]# prompt and you'll be then prompted by the script to answer Yes or No as to whether you want to stop services and run the script. Once you answer YES, the script will stop the appropriate services, back up the files that will be modified, and then apply the modification(s) to the necessary files before restarting the associated services. Depending on the speed of your ESX host and the underlying disk drives this process can take 20 minutes or longer. Be patient. You'll then see output similar to this:
Once the first script completes, we can now run the second script by typing: python remove_log4j_class.py and hitting enter. You will be prompted to answer yes or no to continue. Once you type y and hit enter, the script will run with screen output showing files being backed up before modification, and then you will be returned to a command prompt with a message containing the date and INFO main: Done. You can verify that the script worked successfully by typing: python remove_log4j_class.py -r which should then echo that the list of vulnerable files is now empty. Script three (added 12/21/2021) named vc_log4j_mitigator can be run with the command python vc_log4j_mitigator.py -r with the -r indicating running the script in "dry run" mode that checks to ensure all changes are made, without actually modifying any of the files. NOTE: If you are running vCenter Server Appliance 6.0U3j, the remove_log4j_class scripts may not work. You will have to manually remediate using the steps outlined in the 87081 knowledgebase article.
Once you verify that your vCenter is working properly and that all dependent services are also functioning as they should, you can then delete the VM snapshot that we created earlier. This is important due to the fact that a long-running snapshot file can become rather large and not only slow down your vCenter server but present many other problems later such as exhausted disk space.
Here are the links to the two VMware KB articles. Each has the python script(s) attached on the right-hand side of the screen in the block labeled Attachments:
Here's a link to the VMSA-2021-0028 Advisory that is continuously updated:
If you want to use SCP instead of SFTP to copy files to your vCenter appliance, here are the instructions on changing the default shell from appliancesh to bash. Just remember that once you're done copying the files via SCP, you'll need to change the default shell BACK to appliancesh.
And, last but not least, here's the link to the article on how to reset the root password of your vCenter appliance if it expires. Make sure that IF you have to do this you then log back into the vCenter appliance admin console (port :5480), go to the Administration menu, go to Password expiration settings, click EDIT, and set Password Expires to NO.
*note1: Tuesday 12/21/2021 - Added vc_log4j_mitigator.py script reference due to it being added to VMware KB 87081.
VirusTotal is a website that was originally created by the Spanish security company Hispasec Sistemas. Launched in June 2004, it was acquired by Google in September 2012. The company's ownership switched in January 2018 to Chronicle, a subsidiary of Google. VirusTotal aggregates many antivirus products and online scan engines called Contributors. In November 2018, the Cyber National Mission Force, a unit subordinate to the U.S. Cyber Command became a Contributor. The aggregated data from these Contributors allows a user to check for viruses that the user's own antivirus software may have missed, or to verify against any false positives. Files up to 650 MB can be uploaded to the website, or sent via email (max. 32MB). Anti-virus software vendors can receive copies of files that were flagged by other scans but passed by their own engine, to help improve their software and, by extension, VirusTotal's own capability. Users can also scan suspect URLs and search through the VirusTotal dataset. VirusTotal uses the Cuckoo sandbox for dynamic analysis of malware. [Source: Wikipedia]
Windows servers running the DHCP role store the active copy of the DHCP database in %SystemRoot%\System32\DCHP directory. By default, the DHCP service backs up the database every 60 minutes to %SystemRoot%\System32\DCHP\backup. However, in the event of a DHCP server crash, DHCP database corruption, or a malware event, you might not be able to access this folder to perform a restore operation. You can add another layer of protection by setting up a scheduled task to back up the database to a network share, or you may simply opt to set up a task to copy the DHCP\backup folder to an alternate location at a scheduled time daily. The Powershell command to back up the database is as follows:
Backup-DhcpServer -path C:\DhcpBackupFolder
This command will create a sub-folder called NEW containing the database backup files and a file called DhcpCfg in the root of the target backup folder. You need both the config file and the files in the backup folder in order to restore.
If you should ever have to restore your DHCP server from your powershell-driven backup process, you will first need to copy the config file and "new" subdirectory contents to the "%SystemRoot\System32\DCHP\backup" folder before the DHCP restore operation will function properly.
If you would rather use Powershell instead of the GUI for the restore, the command is as follows:
Restore-DhcpServer -ComputerName "ServerNameHere" -path "C:\Windows\sytem32\dhcp\backup"
Do you know what's NOT intuitive to use? The IBM/Lenovo ToolsCenter Bootable Media Creator. First of all, it's a command-line tool for creating a GUI that updates the firmware on an IBM/Lenovo System-X/xSeries server. So, here's a brief description of how to use this not-so-user-friendly utility.
First, head over to the download page to get the executable:https://lnkd.in/dHFfGbcc
Now, copy the executable into its own folder, for example: C:\BOMC
Next, open a command-line (CMD.exe) as administrator and navigate to the aforementioned folder.
C:\bomc>ibm_utl_bomc_9.41_windows_i386.exe –help (for extraction of help text)
To Create Bootable USB Key:
C:\bomc>ibm_utl_bomc_9.41_windows_i386.exe --usbkey=e: --local=c:\bomc -m 7915 --function=uxspi,dsa --latest
--usbkey= is the drive letter path to the USB key
--local is the folder where you want the driver files downloaded
- m is the model number of the IBM/Lenovo system being upgraded
-- function indications what functions you want loaded onto the image in this case, the uxspi GUI for Update Xpress, and the DSA diagnostics tool GUI
–latest retrieves the latest files from the web
There are options for creating a bootable ISO file as well.
If you run into problems with firmware upgrades failing, move the later versions of the driver for the adapter (for example the raid controller) to a sub-folder so that the UXSPI utility only sees the oldest available firmware, it will then use that version by default. In some cases you may have to go to FixCentral and get an older version of firmware in order to do a step-upgrade to newer versions one at at time. If this is the case you will have to reboot the machine between each firmware upgrade step
There are times when you might need to issue a self-signed SSL certificate for local intranet web applications, especially when the application vendor includes a self-signed SSL certificate for their application, but not for your specific internal domain. The process of creating a self-signed certificate has become much easier lately.
One option is to download and install Git Bash for Windows (https://git-scm.com/downloads). Git-2.37.1-64-bit.exe is the latest version as of this posting, which includes the OpenSSL library. If you use Git Bash to create an SSL certificate, the key is to make sure you put "winpty" in front of the OpenSSL command in the shell, or the command will lock up the terminal window.
Example: winpty openssl genpkey -algorithm RSA -des3 -out c:\folder\my-private-key.pem -pkeyopt rsa_keygen_bits:4096
A second option is to use the New-SelfSignedCertificate commandlet within Windows Powershell (version 5.1 or later).
New-SelfSignedCertificate -dnsname myserver.domain.local -notafter $3year -CertStoreLocation cert:\\LocalMachine\My
*note1: the -notafter switch is not valid on versions of Windows Server prior to 2016. Also, the cert:\\LocalMachine\My stores the new self-signed certificate in the signed-on user's local certificate datastore
An output similar to below will be shown. Note the thumbprint as you'll need to use that in the next command.
Next, you'll need to create a .pfx file.
$CertPassword = ConvertTo-SecureString -String "passw0rd!" -Force -AsPlainText
Export-PfxCertificate -Cert cert:\\LocalMachine\My\\54005B7DB6DC641F9EF982BACD9A8CBEB1D2E15F -FilePath "C:\\PutAFolderNameHere\\myserverHttpsCert.pfx" -Password $CertPassword
You'll need to follow the application vendor's instructions for installing your new SSL cert, or in the case of an internally-developed application, install the SSL cert into IIS or Apache using instructions found elsewhere.
After creating the .pfx file, you'll need to import it into your PC's local root certificate store. Go to the Windows Search bar and type MMC (Microsoft Management Console), run as administrator. Got to File -> Add or Remove Snap-ins -> Certificates and click Add and then OK. Next, Navigate to the Certificates (Local Computer) add-in, then right-click on Trust Root Certification Authorities -> All Tasks -> Import. In the Certificate Import Wizard click Next, Browse and go to the folder where you output your PFX file, select the file, click Open, Next and then finish the wizard.
Now, when you browse to your intranet web application you should not get the "Your Connection is Not Private" error, but rather it will now reflect it as an SSL site.
Akzium team blog
Akzium, LLC | 601-841-2499 . firstname.lastname@example.org
Copyright 2011-2022, Akzium, LLC. All rights reserved.