Adding Default Gateway in ESXI via CLI/SSH
To add default gateway in esxi server using CLI/SSH, login to ssh & issue below cmd’s
#Check Current Gateway , if any
#CMD 1
esxcli network ip interface ipv4 get
#CMD 2
esxcli network ip interface list
#CMD 3
esxcfg-route
>>> VMkernel default gateway is 0.0.0.0
# Add Default Gateway 10.0.0.1 (change as per actual)
esxcfg-route -a default 10.0.0.1
>>> VMkernel default gateway set to 10.0.0.1 #Verify if route have been added. esxcfg-route >> VMkernel default gateway is 10.0.0.1
VCSA 6.7 | 503 errors while accessing vcsa portal
503 Error- Solution # 1
While accessing VCSA via web browser, we faced following error
Solution#1:
Login to VCSA by SSH, & issue below cmd’s
shell.set --enabled true
shell
service-control --status --all
if you see services in STOPPED: section, then issue below cmd to start all services
service-control --start --all
Now try to access the VCSA via browser & it should work fine.
503 Error- Solution # 2
After logging to VCSA via SSH , and issue df -h, you see SEAT partition full or if you login to the management panel you see following,
If you are seeing seat partition getting near full, its better to extend that disk via ESXI server & then extend it using VCSA cmd’s. Also you can truncate the DB to remove old data logs.
Solution#1:
Login to VCSA by SSH, & issue below cmd’s
shell.set --enabled true
shell
# Login to Database
/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres
This will display the top 30 largest tables within the vCenter Server database
SELECT nspname || '.' || relname AS "relation", pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size" FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind <> 'i' AND nspname !~ '^pg_toast' ORDER BY pg_total_relation_size(C.oid) DESC LIMIT 30;
You will see tables as per following
VCDB=# SELECT nspname || '.' || relname AS "relation", pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size" FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname NOT IN ('pg_catalog', 'information_schema') AND C.relkind <> 'i' AND nspname !~ '^pg_toast' ORDER BY pg_total_relation_size(C.oid) DESC LIMIT 30;
relation | total_size
---------------------+------------
vc.vpx_event_arg_8 | 145 MB
vc.vpx_event_arg_2 | 143 MB
vc.vpx_event_arg_1 | 143 MB
vc.vpx_event_arg_92 | 143 MB
vc.vpx_event_arg_10 | 143 MB
vc.vpx_event_arg_11 | 143 MB
vc.vpx_event_arg_16 | 143 MB
vc.vpx_event_arg_13 | 142 MB
vc.vpx_event_arg_3 | 142 MB
vc.vpx_event_arg_90 | 142 MB
vc.vpx_event_arg_9 | 142 MB
vc.vpx_event_arg_4 | 142 MB
vc.vpx_event_arg_91 | 142 MB
vc.vpx_event_arg_15 | 142 MB
vc.vpx_event_arg_14 | 142 MB
vc.vpx_event_arg_87 | 142 MB
vc.vpx_event_arg_5 | 142 MB
vc.vpx_event_arg_12 | 142 MB
vc.vpx_event_arg_88 | 142 MB
vc.vpx_event_arg_89 | 142 MB
vc.vpx_event_arg_6 | 142 MB
vc.vpx_event_arg_21 | 142 MB
vc.vpx_event_arg_22 | 141 MB
vc.vpx_event_arg_23 | 141 MB
vc.vpx_event_arg_17 | 141 MB
vc.vpx_event_arg_7 | 141 MB
vc.vpx_event_arg_19 | 140 MB
vc.vpx_event_arg_24 | 140 MB
vc.vpx_event_arg_18 | 140 MB
vc.vpx_event_arg_20 | 140 MB
(30 rows)
Now by seeing the table names, you can Truncate the large tables individually
For example:
truncate table vc.vpx_event_arg_8 cascade;
Keep cleaning as per disk space requirements, then Exit the vCenter Server Appliance database by running this command:
\q
Verify the space is reclaimed by running the df -h command.
Preferably restart the VCSA services or restart the VCSA server so that alerts can clear up quickly
Read below vmware guide for more information
https://kb-uat.vmware.com/s/article/2119809
VCSA DB Tables Automatic cleanup via SCRIPTING ! Workaround using BASH SCRIPTING
First, Connect to VCSA via putty, & set shell
shell.set --enabled true
shell
Now create two files, one is the bash file that will execute an sql query, & then the second file which will hold the sql query.
First the SQL query file
vi /root/dbcleanup.sql
& paste following data in it.
DO
$$
DECLARE
rec record;
BEGIN
FOR rec IN
SELECT *
FROM pg_tables
WHERE tablename ~ '^vpx_event_[0-9].*'
ORDER BY tablename
LOOP
EXECUTE 'TRUNCATE TABLE '
|| quote_ident(rec.schemaname) || '.'
|| quote_ident(rec.tablename) || ' CASCADE';
END LOOP;
END$$;
DO
$$
DECLARE
rec record;
BEGIN
FOR rec IN
SELECT *
FROM pg_tables
WHERE tablename ~ '^vpx_event_arg_[0-9].*'
ORDER BY tablename
LOOP
EXECUTE 'TRUNCATE TABLE '
|| quote_ident(rec.schemaname) || '.'
|| quote_ident(rec.tablename) || ' CASCADE';
END LOOP;
END$$;
Now the BASH script that will actually trigger the above dbcleanup.sql query file
vi /root/dbcleanup.sh
and paste following lines in it
#!/bin/bash
#> /var/mail/root
DT=`date`
echo "$DT - DB process started ..." >> /var/log/db.log
# to get the PGPASSWORD, pull it from this cmd > "cat /etc/vmware-vpx/embedded_db.cfg"
export PGPASSWORD='w@x!8q6FB1HvmU9='
/opt/vmware/vpostgres/current/bin/psql -U vc -d VCDB postgres -f /root/dbcleanup.sql -h /var/run/vpostgres
DT=`date`
echo "$DT - DB process ended ..." >> /var/log/db.log
(-h /var/run/vpostgres syntax was necessary to execute the psql CMD, it took 1 day to resolve, without this, you will get psql connect error)
Press :wq to save & exit VI editor.
Now schedule it to run every hour or as you like
crontab -e
Press i to go in edit mode and paste following
# to execute the table cleanup script for every 5 minute,
*/5 * * * * /root/dbcleanup.sh
# to execute the table cleanup script every HOUR,
@hourly /root/dbcleanup.sh
Now press Escape, and then :wq which will result in save & exit the vi editor.
Congrats ! all done.
This was written in particular to VCSA ver 6.7 only !
Syed.Jahanzaib
ESXI Server: ‘SEL Fullness’ message
Under ESXI, or VCENTER , I was seeing following alert ..
System Management Software 1 SEL Fullness: Log almost full
Solution # 1
In vSphere 6.7 , Navigate to following and reset event log
Monitor > Tasks and Events > Hardware Health > SYSTEM EVENT LOG > RESET EVENT LOG
Solution # 2
Enable SSH on that esxi host, and connect to it via ssh client & issue below cmd’s
localcli hardware ipmi sel clear
/etc/init.d/sfcbd-watchdog restart
services.sh restart
For few minutes it will show disconnected in vcenter, after few minutes, Refresh the Webclient & related errors will be cleared.
Alhamdolillah!
Updating ESXI 6.5 to 6.7u3 from the CLI
Recently I did some major updates in our virtual infrastructure including ESXi (ver 6.5.0, 10719125) , Vcenter & Veeam (9.5.4.2866) . Things were updated in following order
- Veeam B&R upgraded to ver 9.5.4.2866
- Vcenter upgraded to ver 6.7.0.40000
- ESXI hosts upgraded to ver 6.5.0, 10719125
For ESXI update from 6.5 to 6.7u3. In the past I always use Installer CD/USB to upgrade from older esxi to new version, but for this approach, I have to compromise on my holidays or sit very late in office. This time time I took another approach and upgraded all the esxi hosts one by one on sundays remotely from the home using offline bundle installer & esxi CLI method. This is how I accomplished it.
I first downloaded the 6.7 update offline bundle from the Lenovo site (since we have all the IBM/Lenovo brand servers therefore I selected this option to avoid any hardware driver issue).
https://vmware.lenovo.com/content/custom_iso/6.7/6.7u3/
Afterwards I uploaded this offline bundle zip file to Esxi datastore, then logged in to esxi host via SSH, and issued
esxcli software vib install -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip
Note: make sure to change datastore and filename as required.
It took few minutes, once I saw SUCCESS message, I completed the process by simply rebooted the ESXI host by cmd
reboot
Better approach is to update rather than install
esxcli software vib update -d /vmfs/volumes/5d0cf64f-a83e7c86-6a4d-40f2e922c64a/Lenovo_Offline_Bundle_VMware_ESXi_6.7.0.update03_14320388_LNV_20190920.zip
Difference between VIB update and VIB install
Excerpt from “https://communities.vmware.com/thread/435959”
To install or update a .zip file, use the -d option. To install or update a .vib file use the -v option.
Using the update command is the recommended method for patch application. Using this command applies all of the newer contents in a patch, including all security fixes. Contents of the patch that are a lower revision than the existing packages on the system are not applied.
Using the install command overwrites the existing packages in the system with contents of the patch you are installing, including installing new packages and removing old packages. The install command may downgrade packages on the system and should be used with caution. If required, the install command can be used to downgrade a system (only for image profiles) when the –allow-downgrade flag is set.
The install method has the possibility of overwriting existing drivers. If you are using 3rd party ESXi images, VMware recommends using the update method to prevent an unbootable state.
Check esxi version from CLI
esxcli system version get
Product : VMware ESXi
Version : 6.7.0
Build : Releasebuild-14320388
Update : 3
Patch : 73
Will update more
Regards
Syed Jahanzaib
24.910234
67.058074
Syed Jahanzaib - Personal Blog to Share Knowledge ! 