Post | Handle leakage? | Tech Off | Forums

Here's some code I wrote that sets the right checksum field value in a PE file's header. It's part of a larger program that modifies system files:

(Sfh is a type alias for SafeFileHandle)

// Get handle to a file with read/write access
// CreateFileMapping
// MapViewOfFile
// Check DOS/NT Headers (optional, not needed in this case since if it's gotten this far it's going to be a Win32 PE)
// CheckSumMappedFile
// Close the handles
            
            Sfh    sfh   = null;
            IntPtr hMap  = IntPtr.Zero;
            IntPtr pData = IntPtr.Zero;
            
            try {
                
                sfh = NativeMethods.CreateFile( fileName, FileAccess.ReadWrite, FileShare.None, IntPtr.Zero, FileMode.Open, 0, new Sfh(IntPtr.Zero, true) );
                
                if( sfh.IsInvalid ) throw new AnolisException("Invalid SafeFileHandle: " + NativeMethods.GetLastErrorString() );
                
                hMap = NativeMethods.CreateFileMapping( sfh, IntPtr.Zero, FileMapProtection.PageReadWrite, 0, 0, null );
                
                if( hMap == IntPtr.Zero ) throw new AnolisException("Invalid File Mapping: " + NativeMethods.GetLastErrorString() );
                
                pData = NativeMethods.MapViewOfFile( hMap, FileMapAccess.FileMapWrite, 0, 0, IntPtr.Zero );
                
                if( pData == IntPtr.Zero ) throw new AnolisException("Invalid Map View of File: " + NativeMethods.GetLastErrorString() );
                
                // checking the PE headers for validity isn't required in this scenario
                
                UInt32 oldChecksum = 0;
                UInt32 newChecksum = 0;
                
                UInt32 size = NativeMethods.GetFileSize( sfh, IntPtr.Zero );
                
                // Get the NTHeader
                IntPtr pNTHeader = NativeMethods.CheckSumMappedFile( pData, size, ref oldChecksum, ref newChecksum );
                
                if( pNTHeader == IntPtr.Zero ) throw new AnolisException("CheckSumMappedFile Failed: " + NativeMethods.GetLastErrorString() );
                
                NTHeader ntHeader = (NTHeader)Marshal.PtrToStructure( pNTHeader, typeof(NTHeader) );
                
                // Set the CheckSum, but as this is a new allocation by Marshal it needs to be copied back to the mapped memory
                ntHeader.OptionalHeader.CheckSum = newChecksum;
                
                // this overwrites the old NTHeader @ pNTHeader
                Marshal.StructureToPtr( ntHeader, pNTHeader, false );
                
            } finally {
                
                String errors = String.Empty;
                
                if( pData != IntPtr.Zero )
                    if( !NativeMethods.UnmapViewOfFile( pData ) ) {
                        errors += "UnmapViewOfFile failed: " + NativeMethods.GetLastErrorString() + "\r\n";
                    }
                    
                if( hMap  != IntPtr.Zero )
                    if( !NativeMethods.CloseHandle( hMap ) ) {
                        errors += "CloseHandle(hMap) failed: " + NativeMethods.GetLastErrorString() + "\r\n";
                    }
                    
                if( sfh   != null && !sfh.IsInvalid )
                    if( !NativeMethods.CloseHandle( sfh ) ) {
                        errors += "CloseHandle(sfh) failed: " + NativeMethods.GetLastErrorString() + "\r\n";
                    }
                
                
            }

The problem is, this code doesn't seem to close the handles right. My program's exception log is on Pastebin. The "The handle is invalid" exceptions occur at random intervals.

At the end of everything, my process' handle count is at about ~200, so it's not like it's obviously leaking them. But is there anything in here that isn't done right that could fail?

Post permalink