Wednesday, February 19, 2014

Fixing Windows Registry broken permissions through SysInternals

If you are a Software Developer or Systems Administrators, and you are not familiar with Sys Internals, then you should read this article, because although I address a very specific problem here, I'm completely sure that what I'm going to tell you, will help you for sure during your path in the field, specially when dealing with Windows.

It was being a very long day, setting up all the .NET solutions, Visual Studio 2008, Visual Basic 6.0, COM Libraries registration, SQL Server, Git for versioning and source control, but I did it all.

After, I sat at my sofa and said to myself, "Nice Work"! But just before turning my machine off, I remembered that I didn't add that reference to MSXML v4 to my VB6 project... So, to prevent having nightmares about that, I just connected to the server and decided to add the reference, but it turns out that, I faced this instead:
That was about to spoil my night... but 2 hours later I found the solution, so I decided to share it with you, hoping that no one else, needs to pull off hair from their head cracking this up.

I hope you are familiar with SysInternals, a couple of tools developed by Microsoft employees to help you out diagnosing and dealing with the "guts of the beast" (Windows).

Let's start by analyzing the error message "Error accessing the system registry", well this doesn't say much does it? Or it actually says enough if you know the right tools. Notice this is a big problem, so we need to use Divide and Conquer or you will not solve this in an acceptable time range :) .

  • So VB6.exe says: "Error accessing the system register." right?
  • Now to understand what VB6.exe is accessing in the register, we need to use the sys internals tool: Process Monitor (just download it to your desktop and let's proceed);

    This tool will show you what register keys are being accessed by the applications running in your machine, although to find what we want we will filter the information, adding the following rules:

    - exclude: process name is not VB6.exe
  • After doing this, go to your VB6.exe and try to open the References window again. You will now see a lot of RegOpenKey operations in Process Monitor, that's it, we are getting closer to the origin of the problem;
  • If you look closer to the RegOpenKey entries in Process Monitor, you will see that we have a column called 'result', if you filter this column by 'ACCESS DENIED' (I found this by inspecting all values in this column), you will trim your results to just a few entries, and those entries are more easily handled because they represent the register keys that your VB6.exe is trying to read without success, and that's why you see that error message!

So now we just need to fix the permissions to that register keys right?! Well, it's not that easy I must say, because a register key can contain sub keys, and if the permissions are really mess up, you will not be able to change the permissions of the keys, and even if you are able to fix some of the permissions, do this for all the keys and their sub keys it's impractical in general.

In my case I ran regedit from a command line in administrator mode, and not even this way I was able to change the permissions of the register entries (keys)! Why? Because when the ACL's (Access Control Lists) of windows register entries get broken, you will not be able to change them, unless you are the SYSTEM account! An account that you can't use directly and is like GOD mode in Windows OS! 

Because of this, the first solution may seem to throw the Server out off the window :], but there is something we can do to fix this, which is more wise than the first solution. What if, we have a way to run a process as the SYSTEM account? 

Hum, there is one... sys internals has a tool for that called PsExec (psexec).
You can run it from the shell (cmd) like this:

psexec.exe -i -d -s cmd

This will open another command line window that executes commands as the SYSTEM account, so now we are ready to run the ultimate tool that will get our register keys fixed for good, the SubInACL. SubInACL is a command-line tool that enables administrators to obtain security information about files, registry keys, and services, and transfer this information from user to user, from local or global group to group, and from domain to domain.

Note that if you run this tool just as an administrator you will not have permissions to fix all the entries with broken ACL (permissions), you need to run it as SYSTEM to succeed on this task.

Now to run subinacl.exe I recommend you to create a batch file (.bat), for my case I just wanted to run the script for the register containers "HKEY_CLASSES_ROOT\TypeLib" so my script aims to this, but you can point to other containers.

My script to reset the permissions looks like this:

#navigate to the folder where subinacl is installed
cd /d "%programfiles% (x86)\Windows Resource Kits\Tools"

#this fixes the permissions for all subcontainers of 'HCR\TypeLib'
subinacl /errorlog="subinacl-debug.txt" /subkeyreg HKEY_CLASSES_ROOT\TypeLib 
         /grant=administrators=f /grant=system=f /grant=users=r /setowner=administrators

#this fixes the permissions for the container 'HCR\TypeLib'
subinacl /errorlog="subinacl-debug.txt" /keyreg HKEY_CLASSES_ROOT\TypeLib 
         /grant=administrators=f /grant=system=f /grant=users=r /setowner=administrators

#this opens the log file, containing debug info about the fix operations
notepad subinacl-debug.txt

Now you should call this script from within the command line opened before as SYSTEM. After running this, all you permissions with broken ACL's in Process Monitor should be fixed and when going to References Window now, inside Visual Studio 6 you should be able to see your references again without the initial error!

And that's it, did you see how we solve this? By dividing and conquering the solution?
So my advice is, when facing a problem like this, always try to look for all the details and analyze everything carefully. And above all, remember that, we should not blame Microsoft for this.

The reason for this kind of things is related to all the automatic stuff that happens under the hood when someone decides to start installing everything they remember in the machine, and like everything, sometimes things go wrong.

Windows does it's best to bring the joy of "next, next" experience to their users, but sometimes, this is not possible and you must know your machine to know how to fix the problems yourself. Hopefully this tends to be a rare situation, if that's not your case, and you face problems with your server every week then something are going wrong between the keyboard and the chair somewhere in the process :) .

Hope you enjoyed my article and that you got your problem solved.