Stuff I wish I knew earlier

ESX datastore in use but nothing is there!

2011-12-27T14:49:00.000-05:00

I was moving data off a datastore to reformat it as a VMFS 5 filesystem (ok, I'm probably a little paranoid about upgrading). After moving everything off, I still could not delete it as I was getting a 'resource in use' error. Browsing the datastore didn't show anything. I made sure that HA wasn't using it. After poking around more, I used the 'maps' feature in vCenter to see what it thought was connected and 'lo there was a VM mapped to that datastore. I proceeded to verify that there wasn't a reference via the CD or floppy device, all the disks were referenced, etc.

What took me too long to figure out was that the VM had a couple of snapshots and those snapshots had references to the datastore. Delete the snapshots and the reference is gone and the datastore can be deleted normally.

EMC VNXe

2011-07-26T03:41:00.003-04:00

The EMC VNXe 3300 is targeted to be a low-end / SMB / branch office type array.

These two documents would definitely have been useful to read before we bought the array

Some things that are rather surprising (even in a low end system) in the current build (2.1.0.14097):

NL-SAS disks can only be configured in a (4+2) RAID6 configuration
By default, hot spares are not configured when you create a RAID6 storage pool. In fact, when you manually create a Hot Spare Pool, the wizard tries to talk you out of it.
When the failed drive is replaced, the rebuilt data on the hot spare is copied back to the replaced disk. I guess this isn't surprising for people more familiar with EMC.
While VMware is one of the target audiences, there is no support for VAAI.
You can't grow or otherwise resize a LUN.
You can't remap a LUN to a different 'Storage Server' (e.g. move the primary SP of a LUN to the other SP).
Maximum LUN size is 1.999TB
The only performance information you get via the GUI is for the SP and that is CPU load, network activity and disk activity.

Are 100 Drobos 98 too many?

2011-05-25T21:32:00.004-04:00

Drobo is running a promotion to give away 100 Drobos at http://www.drobo.com/100drobos. The winner will be chosen based on a presentation on how to best use those 100 Drobos.

At first blush, this is kinda interesting. Who wouldn't want 100 of any gizmo?

Well, in this case, winning may actually be losing.

The Drobos being given away are the 4 bay, USB/Firewire model instead of the rack mount 12 bay or even the 5 bay desktop units.

Disks aren't being provided. So let's say you take advantage of the Drobo and only buy 2 of the cheapest disks. As a random point of reference, Amazon has a Western Digital 250GB drive for $40 (http://amzn.com/B000Q84G5Q). So that means you'll need to spend $8,000 just to put the enclosures to use. Taxes in the US are also likely to be at least that amount, if not more.

The "BeyondRAID" technology that they have sounds neat and seems like something that would be useful but what's kept me from buying a Drobo (for home or for work) is that my view of the value of this technology is drastically different than their view.

Specifically, a 4 bay USB2 enclosure with the technology is simply not compelling at $400. It should be at half the price. If I wanted the eSATA/USB 3.0 version, that's a whopping $800. Twice the price for a faster interface? The NAS version is even cheaper than the eSATA version.

From a performance point of view, I'd want eSATA or even the BeyondRAID technology packaged into a PCIe card that I could put into my own enclosure that would be interesting too.

From a server/rack-mount, again the price is not compelling and what I am looking for in "enterprise" storage is greater manageability, at least 16 drive slots, and, if you want to be cheap storage, actually be cheap.

Service pack backup files

2011-02-18T08:59:00.001-05:00

Remember. disk space is cheap so don't be a doofus and delete the beta service pack backup files 'cause you'll need them to uninstall the beta and reinstall the final release.

On the bright side, I suppose you get to start over with a clean install of sp1...

When is DVI not DVI?

2010-11-18T09:36:00.000-05:00

When it is a dual link DVI.

I just got a Dell U2711 monitor and it looked awful. It looked like there was a DPI or resolution problem. I was wondering if the graphics card was too old or some other driver/configuration problem.

Nope, needed to use a Dual link DVI cable (thanks http://photography-on-the.net/forum/showthread.php?t=857897). How can you tell the difference? Dual link is printed on the cable.

Would have been nice if Dell included in the 'monitor setup' cheat sheet a little note that you must use a dual link cable.

View 4.5 and Win7sp1 beta

2010-10-07T00:59:00.000-04:00

The new VMware View 4.5 client doesn't work with Windows 7 sp1 (beta) and VMware has a kb article about it.

In the 'well, duh, should have thought of it myself' category... setting the application compatibility on the view client to Vista seems to allow the client to operate normally. At least without IE9 installed.

Random Rant: VMSafe at the VM Security Panel at Catalyst 2010

2010-07-30T10:22:00.000-04:00

VMware was given some crap at the panel for providing APIs to allow visibility and control into the operation of guests running in the hypervisor.

The argument against this was it was "too dangerous" and provided "too attractive a target to attack."   The most inane argument was "if you crack the hypervisor, then this gives you unfettered access to the guests." Seriously? If you crack the hypervisor, you are already toast. Try making the argument that "well, sure, the guy got root but I didn't have adb installed so he couldn't have done anything bad after that."

I've been looking forward to VMsafe since it was announced and disappointed by the lack of any apparent activity.

I want a libpcap port which would then enable a whole slew of open source utilities. I want to see system and application profiling and debugging tools built on this. I want to see monitoring and management tools aggregating the rich information that could be mined from this and reporting it back to an analysis engine (so like AppSpeed done right).   But what do we have now, years later? Nothing.

To be clear, I'm not saying that this should be enabled by default for every VM or you can just telnet to port 39558 to start poking around. Don't be stupid but also don't be afraid of something seriously useful (and oh, by the way, your competitors don't seem to have it).

Maybe the main problem was targeting the security weenies. Maybe you should have called it 'VMDebug.'

Random assertion

2010-07-27T20:17:00.000-04:00

Twitter needs de-dupe, in all senses of the word dupe.

esxcli swiscsi nic add - or how not to handle errors

2010-07-09T16:44:00.000-04:00

"esxcli swiscsi nic add"currently has an issue where all errors result in the same message: "Failed to add nic."

You get this error when you specify an invalid vmk device or (more importantly) when the Port Group for the VMKernel device has more than one uplink available. There needs to be one vmnic as an "Active Adapter"; all other adapters must be listed as "Unused". You can not have an adapter in "Standby"or you will get this message.

VMware does have a KB article on this: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009450

Linux, Multipathing, EqualLogic and me

2010-04-09T17:39:00.001-04:00

When is a multipathing problem not a multipathing problem? When it is actually a multiple interface problem.

Mulitpathing with Linux (RHEL5) and EqualLogic. How hard could that be?

Ok., sure the documentation is lacking in useful details but makes up for it by having extraneous words and vague references. Nevertheless, it is good enough to get things set up. However, on the mkfs I start seeing connection errors:

connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4296210335, last ping 4296215335, now 4296220335
connection1:0: detected conn error (1011)
connection1:0: detected conn error (1019)

and I end up being able to replicate this problem by running bonnie++.

A colleague (much smarter than I) found in the packet trace that eth1 was sending on ARP on behalf of eth2. This seemed to confuse the array and the initiator ended up sending a RST to the array which promptly closed the connection. The iSCSI layer wasn't aware that the rug had been pulled out from under it but eventually caught on and logged the message above.

Naturally, it turns out to be a feature not a bug. Perhaps because both eth1 and eth2 are on the same subnet. In order to decline the use of said feature, adding the following to /etc/sysctl.conf works wonders in getting multipathing to work:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

The arp_announce is probably what is really necessary. The arp_ignore seems paired with it on all the other web pages and so when I was doing the "monkey-google, monkey-paste" I put it in and haven't spent the time to understand it fully and test without it.

Everything is great the physical machine is running fine and to double check I replicate the problem on the VM, apply the changes, and notice that everything is working fine. Unfortunately, within a few minutes, the VM is throwing connection errors again but the physical machine is just fine. It looks like on the VM, we also need to change the connection tracking behavior:

net.ipv4.netfilter.ip_conntrack_tcp_be_liberal=1

in order for the connection to be stable. I was informed by the smarter colleague that we have added this to other systems where we've had ssh and NFS issues.

Now if I can only figure out how to get the bnx2i iSCSI offload to work...

Don't forget to restart iscsid

2009-12-22T10:22:00.000-05:00

Yeah, I know, rookie mistake but hey, after making changes to /etc/iscsi/iscsid.conf, don't forget to restart the iscsid for open iscsi on Linux.

This caught me when I was trying to change the CHAP password. It didn't help that adding '-d 99' to 'iscsiadm -m node -l' seemed to indicate it was picking up the new password.

Windows 7 and the LaserJet CM2320fxi MFP

2009-11-25T15:40:00.001-05:00

It is a strange feeling that Windows 7 has "just" gone GA given that I've been running the betas and had access to the RTM builds.

However, this is also resulting in most of the vendors getting around to getting updated drivers out so you don't have to use the Vista ones.

HP released the "full" edition of the drivers at their download site for the CM2320fxi for Windows 7. At this time, there was not a 'drivers only' download. Unfortunately, the setup executables that walk you through installing the printer can't actually find the network printer: neither the DNS name nor the IP address of the printer works.

The workaround is simply to manually extract the exe file using something like 7-Zip and then use the Windows 'Add Printer' process. Annoying but it looks like everything works fine after that. As a plus (or not, depending on your point of view), the other HP stuff isn't installed (order supplies, etc.).

One amusing thing that happened was that I didn't "cancel" when the installer was running the first time before going to 'Add Printers' in windows. I was able to add the printer just fine and it was only later that I noticed the installer was still running behind some windows. Clicking cancel started the install back-out process. On the plus side, the back-out process removed everything it tried to install; unfortunately for me it did 'too good' of a job cleaning and removed the printer I installed. I guess it is like grass: it grows great where you don't want it to but won't grow where you want it to.

eBooks

2009-11-13T20:41:00.000-05:00

Talking about the Samsung units led to the following chain of thoughts...

I liked the touch screen of the units but the screen was too small to do any real drawing or note taking with it and as annoying as the split keyboard was, it was still the fastest text entry.

Even though text entry wasn't great, it was pretty good with Microsoft Reader as a book reader where I could also check email, web browse a bit and run some other things if I needed too.

The iPod touch took over most of those tasks but didn't have Microsoft Reader so I had find another format as most of my eBook reading was from Baen's Webscription but there are ebook apps to make up for the format change. They have some quirks but if you don't have really high expectations they do basically work.

And thinking of format change, Amazon has the Kindle app and that also works pretty well. I was also pleasantly surprised to find that there is a good deal of free content with Amazon Kindle. The easiest way to find that is to go to the genre you are interested in and then sort from lowest to highest price. While there are plenty of out of copyright books, some new books sneak in (probably to encourage you to get the rest of the series).

HP 5101

2009-11-13T20:24:00.004-05:00

I'm pretty happy with the HP 5101. With 2GB of memory and having a ssd, it is pretty zippy. Windows 7 Ultimate installs just fine and the keyboard is large enough that you aren't trying to jam your hands together to type.

However, the high resolution screen really makes the difference. Having the extra 168 pixels (1366x768) down helps keep dialog boxes and other things without scrollbars from getting cut off by the bottom of the screen.

It is very straightforward to replace one of the DIMMs. You just pop the battery out and push further and a panel opens on the bottom where the DIMM can be added. So, if HP is charging way too much for memory, going third party doesn't involve taking apart the machine.

It was a little disappointing the screen isn't bigger. There seems to be a lot of wasted space which could be used by a bigger LCD.

The system is also very smudgy. The trackpad, the shiny black case, etc. A fingerprint reader would also be a nice touch.

Finally, the power brick has an oddness to it. The plug that goes into the unit seems way too long. Since the power comes out the side, this creates a need for extra room by the left side.

Nevertheless, this is a big step forward from a Dell mini9. While the mini9 is a little lighter, the HP 5101 is more usable.

So out of the two generations of the Samsung Q1U UMPCs that I've had and the Dell mini9, the HP is the most useful unit. (The Samsung units were nice but did fall in the category of being either too big or being too small.)

(Edited to fix memory. Should have been 2GB and not 4GB. Thanks Gary)

HP order tracking

2009-08-20T18:06:00.003-04:00

I'm amazed at how much of a lead Dell still has with regards to their internal IT.

When I place an order with Dell, I have one place to look for my order status. It is updated frequently and is usually fairly accurate. I can get email notifications of shipment and all the stuff you'd expect these days.

I placed an order with HP. The email confirmation sends me to an order status web site that never seems to update. Since it was past the ship date on that site and still no tracking information, I had to call (as that was the only suggestion on the web page. I call the number on the web page and the person I finally talked to (after being transferred 3 times with 3 different numbers and hung up on once during a transfer) said they had to check the warehouse and would email me back within 3 hours. That happened as stated but the news wasn't good.

I get an email back and a different URL to check on the order status, this one seems it may actually be updated in which it says that on 8/13 (the day after I placed the order), the ship date was moved back to 9/14.

Too bad Dell doesn't have a good competitor to the HP Mini 5101 at this point.

Update: The 5101 arrived earlier but it still missed the trip I wanted to take it on - by, of course, 1 day. HP was unable to change the shipping to next-day (which would have resulted in coming in on time) but they did refund me the shipping charge since it shipped after they stated it would. FWIW, I still prefer my experiences with Dell.

Zalman ZM-NC2000 and Dell E6500

2009-08-18T19:37:00.002-04:00

I'm annoyed by laptop coolers in principle, like the Zalman NC-2000. Any decent laptop should have been designed with sufficient cooling in mind for mostly normal use. I mean, I don't have to buy an add-on radiator for my car.

I'm even more annoyed that this unit works well for my Dell Latitude E6500. It isn't a low end configuration: the CD-ROM bay has a 1.8" SSD, a 250GB disk drive in the SATA bay, 8GB physmem, T9600 CPU and a SSD in the ExpressCard slot. Nevertheless, I still found it surprising that under some stress in an air conditioned office, the system, sitting flat on a desk, would heat up enough that the machine needed to drop the clock rate down.

After installing Speedfan, I saw the core temperatures top 70C without the cooler. That's just nuts.

With the cooler, there really isn't much noise and the keyboard (or any part of the machine) isn't even getting warm.

So, I'm annoyed it is actually working which means that I'll want to pack it on business trips (and it isn't all that portable...).

HP Color LaserJet CM2320fxi MFP

2009-07-03T03:54:00.002-04:00

Delivery took forever on this printer. Overall it is working fine but the things not mentioned in the brochure are:

The web management interface is extremely limited. You can not restrict printing based on a list of MAC or IPs. I assumed that all HP Business laser printers would have this functionality.
If you want to use legal sized paper in the printer paper tray, the paper tray doesn't sit flush with the printer anymore.

Printing from Windows 7 (build 7100) works fine. No complaints from the Macintosh folks.

One problem I haven't tracked down was when I sent a job as simplex, it still printed duplex.

Dell Latitude E6500 virtualization BIOS option

2009-02-16T10:42:00.001-05:00

With BIOS A11 on the Dell Latitude E6500 with the Intel T9600 CPU, if you want to enable the Intel VT extensions so VMware Workstation can run a 64bit VM, you need to have the 'Trusted Execution' Virtualization BIOS option set to OFF.

If you have it set to ON, the VMware CPU check says that the CPU supports 64bit but is turned off on the BIOS.

An Optiplex and a disk walk into a bar...

2007-07-24T07:17:00.001-04:00

So let's say you have a Dell Optiplex 745 minitower that you'd like to expand by adding another disk. Other than getting another computer, what can you do or watch out for?

The second drive bay is right under the first drive bay. The only fan in the system is the CPU fan above. This may cause some heat problems... See http://support.dell.com/support/edocs/systems/op745/en/UG_en/mt_drive.htm#wp1136008
No SATA cables (data or power) are provided for the second disk
If you want to add additional cooling be aware that since the optical device is also SATA now, the only non-SATA power is to the floppy drive and the leads to the main fan.

On the bright side...

The Zalman ZM-2HC2 disk cooler works in the bay under the optical bay
The Highpoint RocketRaid 2200 PCI Express works in the PCIe x1 bay (and actually works under x64 Vista)

What's an extra 32 bits?

2007-06-21T16:35:00.000-04:00

One may think that now that EMT64 extensions are common in laptop, desktop and server chips that it should be no problem running an 64bit operating system.

(It is somewhat amusing to consider what one is optimistic about and what one is pessimistic, or realistic, about.)

With this brand-new Dell Latitude D830 with 4GB of physical memory, one isn't forced to run a 64bit operating system, but, hey, why not? Got this Vista x64 ISO chewing up disk space on another machine so why not put it go good use?

Well, the answers as to why not vista x64 are currently:

No Bluetooh driver (Dell D360)
Video driver crashes daily; sometimes even more (luckily all this means is that the screen freezes for a second or two and then comes back).
Palm Treo 700p drivers won't load (even though the Palm Desktop installs ok

On the bright side, the SATA AHCI drivers loaded and seem to be running fine

Why it all appears to be magic...

2007-06-14T09:07:00.000-04:00

A while back I installed Windows Vista on a Dell Latitude D810 laptop. Given that Vista's Resource Monitor now provides more information, I noticed that my CPU wasn't running at full speed. What was unfortunate was that this machine was out of warranty and so I had low expectations (which were met...) for Dell Support. Their best suggestion was reinstall XP (which, I'm sure you are not surprised to hear didn't change a thing).

Jump to the present. My other Dell Latitude D810 started running slow after a system board swap. Speedstep tools, cpu-z, and other tools were all reporting that it was being clocked down to about 800mhz.

Jump to the answer. The problem was that the fan above the graphics card was unplugged when the tech replaced the motherboard. After plugging it in, everything worked fine. Running diagnostics on the other machine indicated a similar fan failure.

So given that the operating system was clocking the CPU down because of a fan failure, couldn't there have been a visible system alert that said something? I mean, I can have a tray icon tell me that someone logged in, why can't I have something that says "your fan is broken"? Instead, it all looks like magic stuff is happening in the background....

Shuttle SN25P and RAID5

2005-07-22T18:24:00.000-04:00

It isn't that hard to convince yourself that the Shuttle SN25P supports RAID5. After all, it says it has the NVIDIA nForce4 ultra chipset and one of NVIDIA's pages indicate that RAID5 is now available. Not only that, you click on the NVIDIA RAID link on Shuttle's product page and the NVIDIA page prominently mentions RAID5..

Given the cookie cutter/cut 'n paste nature of some of the reviews, you just aren't sure whether or not someone actually looked at the RAID options or just took someone else's word for it.

After having purchased the unit, installed the latest BIOS (fn25s01o), what have I discovered? The answer is... Nope, no RAID5.

The SN25P has 4 SATA ports and it isn't hard to put a disk drive under the optical drive. It even has a SATA power plug for a disk in that location.

The current BIOS also doesn't have Intel's newish feature of allowing you to take two disks and have a RAID1 set on part of the disk and a RAID0 set on the rest of the disk.

So, to my former self, you would have been better off just planning to put a floppy in that bay and not thinking that you could optimize things by having 3 disks in a RAID5.

Them thar floppies...

2005-07-22T18:02:00.000-04:00

You'd figure that the floppy disk on new computer systems would be as useful as a USB toothbrush or the human appendix: it doesn't really serve a purpose but doesn't really hurt to have around.

Well, I've been proven wrong. The problem comes in when you are installing Windows and the drivers for the disk subsystem are not on that CD. The workaround for this is that Windows will allow you to hit F6 during the initial load and before it actually tries to run, it'll give you the opportunity to insert a floppy disk with the drivers.

As a workaround, the BIOS on new systems will allow you to make a USB flash disk look like a floppy or perhaps you can use a USB floppy drive.

When you see the BIOS option, you figure there is some light at the end of the tunnel but the problem is that the install process needs to access the floppy twice. Once when the installer operating system is loading (so it can see the disks to allow you to partition and format them) and then one more time to copy them to the newly installed hard drive.

What seems to happen is that the BIOS can help you out for the first access. However, once the operating system loads, the BIOS hands control over and now suddenly you don't have a floppy disk in A:\ and so the install bombs.

Unless you were lucky enough to get one of those USB floppy drives that is recognized. (http://www-1.ibm.com/support/docview.wss?rs=0&uid=psg1MIGR-56064).

So, what would I tell my earlier self? Just get the damn floppy drive for now. Hopefully, Microsoft will fix this issue in Longhorn.