The Empirical Manifesto

Script which is meant to run from a directory containing compressed/encapsulated EMCgrabs either from ESX or linux hosts.
#
Removes hostnames (retains domain).
Removes up to three (3) IPs.
Handles those god-awful filenames which contain spaces.
Re-compresses new tarball(s) sans hostname for transmission.
#
sanitize_emcgrab_runme.sh
sanitize_emcgrab_data.sh

God I hate Xf86config and xorg conf files. And we're flush with virtual servers these days. This occurred on a headless physical, but the steps should be the same to change from a forced 1024x768 to 1920x1080:

# init 3 to kill your window manager.
# X -configure to create an xorg.conf.new
# cp xorg.conf.new xorg.conf to overwrite the old with the new.
# init 5 to restart your graphical environment.

I use NoMachine (nx) which has a Display --> Change Settings --> Custom Resolution functionality I used to great success after the above steps.

Ugh.

Linux Mint is fantastic. Until its not. Cinnamon runs into all sorts of issues with its "hardware device driver" snafu, the occasional session bus error (at least when running cinnamon on openSUSE), and apt often renders itself non-functional with the nonsensical error sudo apt --fix-broken install which of course would be awesome if it worked. But it doesn't. It never has. It doesn't even appear to be valid syntax. This variation however, does work:

apt-get -f install

Why the openSUSE online repositories had to go offline today however, is just maddening.

God I hate Xf86config and xorg conf files. And we're flush with virtual servers these days. This occurred on a headless physical, but the steps should be the same to change from a forced 1024x768 to 1920x1080:

# init 3 to kill your window manager.
# X -configure to create an xorg.conf.new
# cp xorg.conf.new xorg.conf to overwrite the old with the new.
# init 5 to restart your graphical environment.

I use NoMachine (nx) which has a Display --> Change Settings --> Custom Resolution functionality I used to great success after the above steps.

Ugh.

Linux Mint is fantastic. Until its not. Cinnamon runs into all sorts of issues with its "hardware device driver" snafu, the occasional session bus error (at least when running cinnamon on openSUSE), and apt often renders itself non-functional with the nonsensical error sudo apt --fix-broken install which of course would be awesome if it worked. But it doesn't. It never has. It doesn't even appear to be valid syntax. This variation however, does work:

apt-get -f install

On each my computers with a desktop (2xWindows, 1xOSX, 2xLinux) resides five (5) browsers (Edge, Opera, Vivaldi, Chrome & Firefox) and on each browser resides 70+ tabs. Given memory requirements, I keep these cultivated tab-farms in browser, but only run one browser at a time. Occasionally I will need to pull up another 70+ tab-farm browser to find some information, but keeping it up longer than a few moments is a dangerous game as I never kill my current browser. When my current browser does eventually crash (they all eventually crash), then and only then do I restart a previous browser as a new current default.

This is the way.

For reasons I won't go into here, I increased the capacity of an iSCSI LUN to make more room in a handful of logical volumes. Awkwardly, the parent volume group lost its goddamn mind.

I can't recall if I've ever run into this specific scenario or not before, but I doubt it because I rarely mess around with SAN. Well, until recently anyway. Suffice it to say I was stuck until I could find away to dynamically resize the volume group. Turns out you have to use pvresize as in:

pvresize /dev/sdb

I cannot remember now if I had activated or deactivated the vg prior to doing so, but at any rate it worked.

Either I was always searching for the wrong words or combination of words on Google - OR - the information is so humiliatingly basic as to not be helpful to anyone who already understands SAN topology, I present to you this, my masterpiece of trial and error (basically clinking every goddamn thing I could, in every conceivable combination until something worked.) Also, I'm the only person on the planet not running vCenter which makes it look like it could be configured verbally, "Alexa, map my SAN LUNs to ESXi."
"No problem, Eric. Done."

SO YOU WANT TO ADD SAN DISKS TO YOUR ESXi HOST LOL

ESXi HOST - ESXi 6.7u3:
12x48 PowerEdge R610 with a single 16GB dual-port HBA

SAN HOST - FreeNAS 11.3u2:
32x16 PowerEdge R730 with a single 10GB quad-port HBA

In FreeNAS ensure:

Sharing --> Block Shares (iSCSI) --> Associated Targets

Contains the Target/Extent mapping you wish to use, else even with the above the VM will not be able to see the boot disk.

Sharing --> Block Shares (iSCSI) --> Extents

For ESXi ONLY ensure 512 block size and DISABLE physical block-size reporting!

Lastly, your iSCSI disks may appear as NORMAL,DEGRADED on your ESXi GUI. This is due to there only being a single-path to your SAN disk.

In ESX ensure:

Host --> Manage --> Hardware --> PCI Devices --> "[brand] [speed] [port number] network connection" (this is your HBA)

Ensure Passthrough is DISABLED (enabling is apparently awesome if you want ONLY EVER A SINGLE VM to see the SAN via fiber not FCoE)

Reboot the whole entire goddamn ESXi host if you had to toggle passthrough.

Storage --> Adapters --> (vmhbaXX - iSCSI software adapter - online - iscsi_vmk)

Highlight that bitch and whichever is not grayed out (Configure iSCSI or Software iSCSI) click that one.

You have to enter the SAN's copper network IP as a DYNAMIC TARGET (port should be pre-populated with 3260) and when you click {ok} or {next} or whichever verbiage indicates you've completed this task and should move on to the next...

STATIC TARGETS populate with the BLOCK SHARE TARGET NAMES from the SAN :O

We all have our philosophies surrounding system administration, and mine is learning by utilizing RTFM only as a last resort. According to the SUSE Linux Enterprise Server 15 SP1 Deployment Guide:

The boot manager GRUB 2, used to boot machines with a traditional BIOS, does not support UEFI, therefore GRUB 2 is replaced with GRUB 2 for EFI. If Secure Boot is enabled, YaST will automatically select GRUB 2 for EFI for installation.

I tried it both ways, but EFI with Secure Boot disabled seems to be the way to go.

Edit Settings --> Add Hard Disk --> New Raw Disk --> (compare hostname UUID mapping from FreeNAS)

Boot VM from attached optical media and it should jump right into Installation and know right where to install :)

WHEN YOU ADD ANOTHER PATH TO AN EXISTING ESXi:

Storage --> Adapters --> Rescan
Storage --> Devices --> Rescan
Storage --> Devices --> Refresh

Should populate with the new LUN IF THE LUN ON THE SAN SIDE has a unique LUN_ID.

Today we started with a lesson in the three remaining major linux distros and their derivatives:

• RHEL


• Centos


• Fedora


• Debian


• Ubuntu


• Mint


• SLES


• openSUSE

And how they differed in their package managers (RPM & DPKG) and their online package repositories (yum, apt, and zypper). From there we updated her packages, rebooted (because there was a kernel update, which allowed us to talk about the kernel, and more on the shell), and she installed her first program, neofetch which gave us the opportunity to discuss, architecture, uptime, uname, (kernel) and /etc/*release*.

After that we started simply with more, head, tail as a follow-on to the cat command, which led us to pipe (|) and grep then went right into absolute versus relative paths, and surprisingly enough, file ownership and permissions, which she picked up way faster than I expected. I'm just pleasantly surprised at how quickly she's figuring out and running with the little information I give her :)

Set up the NFS share Maproot user to "root" and the Maproot Group to "wheel"
This way when you have the NFS share mounted and you run something as root on the client you are also root on the FreeNAS box and can change the ownership of the directory.

# chmod 1777 /nfsmount <-- rwx +sticky bit

               ,        ,
              /(        )`
              \ \___   / |
              /- _  `-/  '
             (/\/ \ \   /\
             / /   | `    \
             O O   ) /    |
             `-^--'`<     '
            (_.)  _  )   /
             `.___/`    /
               `-----' /
  <----.     __ / __   \
  <----|====O)))==) \) /====
  <----'    `--' `.__,' \
               |        |
                \       /
           ______( (_  / \______
  (FL)   ,'  ,-----'   |        \
         `--{__________)        \/

ME (a unix admin): "We need some unix-y music,"
HER: (with the quizzical look): "What is unix-y music?"
ME (obvs): PSY GOA TRANCE.
HER: ...

psy goa trance starts playing

HER (a connoisseur): "Oh, like Hackers!"

---

We closed the console and I had her log into the box remotely, via secure shell. Or at least, I tried. The username on the new openSUSE box did not match her username in the Mac Mini terminal, so it was then and there I decided we'd just jump right in. Awkwardly, I didn't realize at the time how overwhelming just changing a username would be for a novice.

This necessitated discussion and practical application on the following concepts as we stepped through each field of a password file:

Where usernames are kept on a unix system
How to ascertain where user passwords are located
What is a GID?
What/Where are user groups?
What goes in a comment field?
Home directories
Brief history of the shell and where to define it

Followed up with:

SysAdmin 101 - always copy a file you're going to edit prior to editing it
Why are we changing all these goddamn files for a single username?
Fuck mv, where is the unix rename command?
What is cat and how to use it after you've just edited a file you previously backed up.

And well before I was ready to even introduce it, but by sudden necessity, an introduction to vi. On the plus side, the gamer in her was able to quickly associate the vi navigation keys h, j, k, l with the w, a, s, d character movement keys in video games, something I'd not even ever considered.

She reminded me at the hour-mark that most classes gives breaks after 60-minutes of lecture and labs. Thankfully, we'd just completed our task.

One of my favorite things to do is force-reinstall dbus on an alternate tty due to "could not acquire name on session bus" every time I have to reboot my linux desktop because something, somewhere, is broken and no one, anywhere, knows what or why.

Got about two solid hours of sleep before I realized I wasn't going back to bed. Too much on my mind. So I shuffled down into the server room and began tinkering with several ideas I'd had lately. First, I wanted to build a FreeNAS server. I have these double-walled totes I keep my hardware in. One is filled completely with SCSI cables, another, 80-pin hot-plug SCA hard drives in a variety of sleds, and another one yet filled with a variety of PCB boards. I dug out the only SCSI adapter I have, an ancient Symbios 22802, and untangled a high-density DB68-to-68-pin adapter cable to connect to a 20-year old Sun 711 disk pack.

The disk pack holds six drives, so I pulled a matching set of 73GB 15k drives from their HP sleds, and removed an equal number of 9GB, 18GB, and 36GB drives from their Sun sleds, and swapped them to complete a matched set of drives for the disk pack. Presumably UltraSCSI doesn't require a terminator, but I dug one of those out as well.

Cracking open the largest of the PowerEdge 2950's I discovered its expansion riser was PCIe, and I needed PCIx for the SCSI adapter card. Thankfully the other PowerEdge 2950 is slightly older, and had a PCIx expansion riser in it! Once that was swapped and the Symbios installed, I rebooted the server with the disk pack attached in several combinations but could never get the card to see the drives. Additionally, I lost half of the six drives during the subsequent power cycles as they hadn't been powered on in 15-years. It was only then I checked the FreeNAS matrix, and even if the server had seen the drives, FreeNAS doesn't support the Symbios 22802.

Once I put everything back together I decided to upgrade the RAM in my OpenBSD bastion, an even older Cisco branded DL320. Upgrade might not be the right word, as the RAM I wanted to use was slower (533MHz vs 66&MHz), but significantly more (8GB vs 2GB). I had pulled this ECC RAM from a second, non-working RX2600 HP/UX chassis but despite the two modules being interchangeable, the Cisco server wouldn't boot with it in any combination. I searched for a physical jumper on the motherboard to shunt to no avail.

In all of this, I did find a virtual SAN, specific to our production environment, which runs solely on VMware to emulate SAN connections to virtual servers, so might try that despite the fact I'm not a storage admin. Could be fun.