Monday, January 10, 2011

IIS 6 and the HTTP 401.3 error

I love it when I find a new tool to use, I love it even more when it is really useful and saves me hours of work. Recently, I had the opportunity to try out ProcMon. This is what happened.

Our test web server started returning HTTP 401.3 errors. The cause was quick to find; the permissions on the root website folder had been changed and the IIS accounts were missing. So the fix appeared simple, re-apply the permissions and they will cascade all the way down the tree. I added the local IUSER account but it failed to fix the problem. I spent several hours with the MSDN documents to make sure I had the correct users and groups applied, but to no avail. I could not find a way to return the server to normal operation.

Finding the problem

The next day I felt resolved to find the problem, no more hacking around throwing users at a dialog box. For help I turned to Process Monitor (ProcMon), part of the SysInternals suite of tools. ProcMon is a superb tool for these situations. It collects all activity on the machine, showing a list file, registry and network activity. Importantly for me, it also records the result of the operation.

I fired it up, attempted to load a web page from my browser, and then stopped the trace. Tracing all the activity on a server will produce a metric ton of data; a one minute trace on my PC generates ~300,000 events. For this reason ProcMon has good filtering. You can pick from a list of events and limit by a text value. I chose to filter the list by Result, only showing those which returned ACCESS DENIED.

With the filter applied there was only one event in the list; the IUSER account was trying to access the file from my browser request. Upon checking permissions on the actual file I found that they were different to the parent. All of the IIS accounts were missing. I forced the permissions down the tree and IIS started serving pages again.

Not just any tool but the right tool

ProcMon is the star here, without it I would have found the problem but with a lot of guess work and a great deal of time. With ProcMon I could see exactly what was happening when IIS tried to serve the page. Being able to see what happens at the core of a system is essential to fault finding and having the right tool is infinitely time saving.