Wednesday, December 10, 2008

WMIC PRODUCT Returns Invalid Class Error 0x80041010

To say that I use WMI (and WMIC) on a daily basis is an understatement. It saves me so much time, but I did run into this bug which I wasted a little time on.

I remembered that there was a WMIC alias called PRODUCT which returned a lot of useful data about what software was installed on the Windows Server 2003 instance. However when I changed jobs from one organisation to another, this function no longer worked for me. I would get an error just like this one:

C:\WINDOWS\system32>wmic product
Node - Server1
ERROR:
Code = 0x80041010
Description = Invalid class
Facility = WMI

The solution was very simple. It turns out that the Windows Installer information is not available in WMI until you install an optional Windows component called "WMI Windows Installer Provider". This component can be installed through "Add/Remove Windows Components"... look for the "Management and Monitoring Tools" group.

If you plan on deploying this to a large fleet of servers, I would recommend you learn about a tool for enabling Windows components in an automated way. The Microsoft tool to do that is called SysOCMgr.exe and is described here. I might right some detailed stuff about SysOCMgr.exe later, so watch this space.

Monday, October 13, 2008

BIOS Serial Number as Computer name in Sysprep image

At a previous job about 18 months ago, I was asked to create a single SOE image that worked on many types of hardware, and was customizable to suit one of four organisations depending on which button the user clicked. It was an awesome challenge and I learned a lot from the experience.

One of the requirements of this Windows XP image was that the computer name of the machine had to be equal to that of the machine's serial number. That is, the hostname, and domain computer account, had to be exactly what your would yield if you used the WMIC command:

C:\> WMIC BIOS GET SerialNumber

I remember that this was quite difficult to achieve, and (for once) Google was of little assistance. There were a few hack 's suggested, like running a script on first start that would pull the value from WMI. But I was after a truely transparent experience which no user could stuff up, and which required no user intervention.

I did meet my deliverables on that project, and eighteen months down the track, I am writing this blog entry to you because it just occured to me that I probably should have shared this at the time to hopefully save people some time.

Setting the computer name to the machine's serial number in a sysprep build


1. Edit your sysprep.inf file and set the following value:


[UserData]
ComputerName=*COMPUTERNAME*

2. Copy and paste (and feel free to re-engineer my poor code) this into a file called GetSerialNumber.wsf


<job id="SetNameToSerial">
<script language="VBScript">
'Get Serial number from BIOS
Set objWMIService = GetObject("winmgmts:impersonationLevel=Impersonate}!\\.\root\cimv2")
Set colItems = objWMIService.ExecQuery("Select * from Win32_BIOS",,48)

FOR EACH item IN colItems
compname = item.SerialNumber
NEXT

'Edit sysprep.inf file
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.OpenTextFile("sysprep.inf", 1)
strText = objFile.ReadAll
objFile.Close
strNewText = Replace(strText, "*COMPUTERNAME*", compname)
Set objFile = objFSO.OpenTextFile("sysprep.inf", 2)
objFile.WriteLine
strNewText
objFile.Close
</script>
</job>



3. Create a Winbom.ini file in preparation for a factory-mode sysprep (with mini-setup)

4. Use the [OEMRunOnce] secion to execute the script... The Winbom.ini file will look something *LIKE* this (I lost my copy!.. and CBF testing it):



[Factory]
Reseal = Reboot
ResealMode = Mini
WinbomType = Factory
UserName = Admin
Password = Password
[OEMRunOnce]
"Computername", "C:\windows\system32\wscript.exe GetSerialNumber.wsf"

5. You're done!

Now just do a factory mode sysprep.

Appologies that this probably wont work out of the box, as I havn't looked at this kind of stuff for 18+ months. I hope it helps. I'm sure this has been achieved by others, but when I needed the method, nobody had documented. I hope I've given you enough info to point you in the right direction. Good luck.

P.S. If you see people on the forums having trouble with this, send them my way :D

Saturday, July 12, 2008

Has MaxFreeTcbs been obseleted?

I'm currently studying the effects of various settings on a file servers performance. One of the TCP/IP registry options that is documented (scarcely), is the MaxFreeTcbs setting. Supposedly this setting is used to decide how many Transmission Control Blocks (TCB) to pre-allocate from non-paged pool. In theory this also has the effect of limiting the amount of concurrent TCP connections the server can sustain.

I loaded up poolmon, and noticed that each time a TCP connection was negotiated, a 456-byte allocation was made from the non-paged pool using the pooltag TCPT. There appeared to be a few TCBs pre-allocated, but only about 5. After that, they were allocated when the TCP connection was created.

What I noticed on a Windows Server 2003 R2 server is that this MaxFreeTcbs setting now appears to have no affect. I set the value to 10, and still able to create 500 simultaneous connections with a tool called PCATTCP. Each of them matched up to an allocation on the TCPT pool tag.

Interesting, nothing mentioned anywhere about this. Also, many of the TCP performance guides still make reference to this key.

Has anyone else run into this? Is this a change to the way TCP/IP works in the Scalable Networking Pack?

-M

Saturday, May 3, 2008

Broadcom Driver Woes

Shortly after deploying the Broadcom NetXtreme II v3.7.19 driver for the 5708 chipset to our Metaframe environment, the users were not happy campers. There ended up being two severe problems that we noticed in this particular revision of the drivers. Today I'm going to describe problem number one, the TCP Offload Engine feature (TOE).

Now we actually ran into this issue 3-4 months ago, so I don't remember what version of the driver we had before 3.7.19.

Once updated, everything appeared fine. Ping tests showed connectivity and expected latency. A traffic simulation tool PCATTCP showed that throughput was being handled as expected; the servers were stable.

The next day when users connected to theses Metaframe servers, they started complaining almost instantly of an obscure symptom, (they always are) slow response to operations in Outlook. For the record, we used Outlook 2003 clients connected to Exchange 2003. Cached Exchange Mode was off, and the driver was only loaded onto the client end.

Now I was baffled to see that the preview pane in Outlook was taking about 5 seconds to update per email, instead of perhaps 200ms. My previous test demonstrated that the NIC driver had achieved far greater throughput than what Outlook was demanding, and that ICMP latency had not increased at all. Surely the NIC doesn't know the difference between ICMP traffic and Outlook traffic! That would defy everything I know of the Networking models.

We ran up Outlook with the /RPCDIAG parameter, and sure enough, RPC latency had shot through the roof about 10-fold. It was also much greater than the standalone client I was using.

I had previously done some research into Microsoft's TCP Chimney Offload from the Scalable Networking Pack and Broadcom's TCP Offload Engine. Could the NIC actually be differentiating between TCP and ICMP traffic thus yeilding conflicting results.

I knew that TCP Chimney Offload (and consequently TOE) could be disabled on-the-fly with one NetSh command:

NetSh Int IP set Chimney Disable

Surely enough within a second, the TCP connections were brought back in host, and all symptoms went away.

The command for determining the offload status for a TCP connection is:

Netstat -t

I'm not 100% sure what broke in v3.7.19 of the driver. The driver just may have been build on a version of NDIS that finally supported TCP Chimney completely, and that TCP offload engine was actually never working previously in our environment. Unfortunately we never got a satisfactory answer as to what went wrong in this driver. If that was indeed expected behaviour for TOE, then there is definitely some tweaking required so that latency-sensitive protocols (RPC in this case) are excluded from being offloaded to the NIC. Some preliminary testing of v4.0.23b of the same driver suggests that TCP latency does increase significantly in an offload state, however Outlook doesn't appear to exhibit the same symptoms. Stay tuned for my investigations into Interrupt Coalescing in the 3.7.19 driver...

Friday, April 18, 2008

Troubleshooting tip for troubled SVCHOST process

Understanding svchost.exe can be useful to help narrow down bugs in Windows services. No doubt you've noticed svchost.exe process instances running in task manager; there are usually 5-7 of them.

The proper name for svchost is the "Generic Host Process for Win32 Services". It's role is to be the host process for dll-based Windows services. So when you see multiple instances of svchost.exe running in your task manager, those processes are actually wrappers for services that are coded into DLL's.

There is not a 1-to-1 relationship between an svchost instance and a service. You can use the tasklist command (or Process Explorer if you prefer SysInternals tools) to see which services are running under any given process. A useful command to find all svchost.exe processes and their associated services is:

C:\>tasklist /svc /fi "imagename eq svchost.exe"

Image Name PID Services
========================= ====== =============================================

svchost.exe 1528 DcomLaunch, TermService
svchost.exe 1580 RpcSs

svchost.exe 1892 AudioSrv, BITS, CryptSvc, Dhcp, dmserver,
EventSystem, lanmanserver, lanmanworkstation,
Netman, Nla, RasMan, Schedule, seclogon, SENS,
SharedAccess, ShellHWDetection, srservice,
TapiSrv, Themes, TrkWks, W32Time, winmgmt,
wscsvc, wuauserv, WZCSVC
svchost.exe 1004 Dnscache
svchost.exe 1160 LmHosts, RemoteRegistry, WebClient
svchost.exe 876 BthServ

Your output will vary slightly, but you will get a good idea about how many services run against each svchost process instance. If you need to expand the service short names (as above) into long names, you can use the sc command. For instance, I don't know what the service BthServ is. Getting scared!

C:\>sc GetDisplayName BthServ
[SC] GetServiceDisplayName SUCCESS Name = Bluetooth Support Service

That sounds legit enough for me :).

So in their design of the Windows operating system, Microsoft decided to group different services into different processes. Why?

Reason 1: At one extreme, Microsoft could have decided to put all services into one process. However if some error were to happen in one service (such as an unhandled exception or access violation) the whole process (all services) would crash. Were you ever told not to put all your eggs in one basket?


Reason 2: The other extreme would be to put each service in it's own process. Well if that were the case I would have another 27 processes on my system. If you have ever studied kernel design you will understand that less processes means less context switches required to keep the system responsive. It's just more efficient to have less processes and more threads. If you find this at all interesting, I highly recommend a book called Operating System: Internals and design principals by William Stallings.


Reason 3: Limiting of the use of permissions. Services are run under different accounts so that they do not inherit privilages they do not need.


I'm sure there are other rationales for why the services are grouped the way they are, but I see those as the big ones.

Now the reason it is important to know how the Service host works is that without knowing it's purpose, it is impossible to debug! There are a few different problems I've had with SVCHOST. Well, actually that's not quite true. I've never had a problem with SVCHOST, it's always been the services running under the svchost process that has been causing problems.

Whenever I've had to debug issues relating to the svchost, it's often resulted in a need to "split" it's roles into other instances. I'm not going to go into the various issues I've had with the svchost, but if you Google svchost 100% CPU , you may find one or two search results on the matter. What I will discuss for you is the technique used to "split" the svchost.

So as I've mentioned, the svchost process hosts one or more services. The standard instances that ship with Windows has a particular configuration that Microsoft are happy with. If you need to, for troubleshooting purposes, pull one or more services into their own host process, here is how you would go about doing that. Lets say the offending service is the WMI service (Windows Management Instrumentation)...


1. Find the short name for the Windows Management Instrumentation service


2. Locate the service in the registry. In this case: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\winmgmt\

3. Observe the current SVCHOST group the service is in by looking at the ImagePath key in the aforementioned registry section. You will see something like "%systemroot%\system32\svchost.exe -k netsvcs". So in this instance, the SVCHOST group is: netsvcs.

4. In this ImagePath key, change the netsvcs group to something different and unique. In this instance I will change it to split_wmi. That is the svchost group I'm chosing to create. Remember that case is critical.

5. Now locate the svchost configuration section of the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SvcHost

6. In this registry section, you will find a Multi-string registry key for each svchost group that exists. So in this case you will find netsvcs. Open that up and take a look. You'll find something like this:




7. This key contains the list of service to start under the netsvcs SVCHOST instance. What we want to do is locate the winmgmt service name and remove that from the netsvcs group. Click OK once you've removed the line.

8. Next you'll need to create a new multi-string key called "split_wmi". This will create a new instance of the SVCHOST with that group name.

9. In the split_wmi key, insert just one line with the service name for the wmi service: winmgmt.

This procedure will become more natural once you've done it a few times. Thankfully you always backup your registry before you start tinkering! :)

I'm running a bit short for time, but if you need any advice tracking down this kind of bug, just leave a comment.

-M