Sam's Code

How to Setup a KD (Kernel Deugger) in Windows With 1394 or Over the Network

2012-03-07T17:23:00.002-08:00

Lets say you are starting to write drivers and need some kernel mode (km) debugging, or lets say you've decided that user mode (UM) debugging using windbg on the host is for sissies. In this post I will show you how to setup a KD.

Assumptions:
You will need two machines: the TARGET machine that you want to debug, and the HOST machine that you will be doing the actual debugging.

First thing you need to do is install windbg on both the target and host. You can find the installer here.

Pick a You KD Method:
Next decide what kind of debugging you want to do. The options are:
NET (i.e. debugging over a TCP/IP network just using NICs) (supported on Win8+),
1394 (supported on WinXP+),
COM (serial) (supported since the dawn of KD), or
USB (2.0 supported on Vista+, 3.0 in Win8+)

Generally the port you use is decided for you based on what OS you need to debug, and what hardware your machines have. I will make it simple, use 1394 (aka firewire) if you can, or if the machines aren't close, net.

If you two machines are next to each other favor 1394. If you are going to be kernel debugging often and don't have 1394 in your machines, buy some cards. 1394 is simple and fast.

If your target isn't close to your debugger machine, use net, short for network, debugging, but note it is a Win8+ feature at the moment. Net debugging is also great for getting someone else to remote debug something. Also in Win8, over 90+% of the mainstream NICs are supported for net debugging; most Intel, Broadcom, and Realtek NICs are supported.

COM is slow, but works assuming your machines have serial ports.

USB might be a choice if your USB controllers support kernel debugging. In my experience, they rarely do. This is especially true when the machine doesn't have 1394 and you can't net debug. You are kind of screwed at this point. The joke is even funnier when you do find a port that does support KD, but it is internally wired to the built in webcam, or doesn't have an external port.

Setting Up a 1394 KD

TARGET

open a command prompt
bcdedit -debug on
bcdedit -dbgsettings 1394 channel:1
- you will have to pass bus params if you have more than one 1394 controler)
- channel can be 1-62
reboot

HOST

plug in 1394 cable into target and host
open a command prompt
kd -k 1394:channel=1

windbg work instead of kd as well

Setting Up a NET KD

TARGET

open a command prompt
bcdedit -dbgsettings net hostip:192.168.1.11 port:50000
- for hostip, put your machine's IP instead of 192.168.1.11
- you can pick whatever TCP port you want as long it is between 49151 and 65536.
It will output something like:
"Key=aaaaaaaaaaaaa.vvvvvvvvvvvvv.yyyyyyyyyyyyy.xxxxxxxxxxxxx"
Save that string in a text file to a thumb drive or network share, you will need it again on the host
bcdedit -debug on
reboot

HOST

open a command prompt
windbg -k net:port=50000,key=aaaaaaaaaaaaa.vvvvvvvvvvvvv.yyyyyyyyyyyyy.xxxxxxxxxxxxx

you can use kd instead of windbg if you want

Debugging Heap Failures

2012-03-02T13:24:00.000-08:00

Sometimes heap failures can be mysterious, but they don't have to be. For instance, today I got this kd (kernel debugger) break.

////////////////////////////////////////////////////////////////////////////
Output of !analyze -v

*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************

Loading symbols for 680c0000 component.dll -> component .dll
Loading symbols for 75fb0000 KERNEL32.DLL -> KERNEL32.DLL
Force unload of C:\Windows\SYSTEM32\user32.dll
Loading symbols for 76680000 user32.dll -> user32.dll
ModLoad: 76680000 767a1000 C:\Windows\SYSTEM32\user32.dll
Force unload of C:\Windows\system32\ole32.dll
Loading symbols for 76560000 ole32.dll -> ole32.dll
ModLoad: 76560000 76672000 C:\Windows\system32\ole32.dll
Debugger Dbgportaldb Connection::Open failed 80040e4d
Database Dbgportaldb not connected

FAULTING_IP:
ntdll!RtlReportCriticalFailure+33
001b:77d912d6 cc int 3

EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 77d912d6 (ntdll!RtlReportCriticalFailure+0x00000033)
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 3
Parameter[0]: 00000000
Parameter[1]: 83b70d40
Parameter[2]: 0000fffd

ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.

EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid

EXCEPTION_PARAMETER1: 00000000

EXCEPTION_PARAMETER2: 83b70d40

EXCEPTION_PARAMETER3: 0000fffd

NTGLOBALFLAG: 0

APPLICATION_VERIFIER_FLAGS: 0

APP: ntkrpamp.exe

LAST_CONTROL_TRANSFER: from 77d924a1 to 77d912d6

FAULTING_THREAD: ffffffff

BUGCHECK_STR: APPLICATION_FAULT_ACTIONABLE_HEAP_CORRUPTION_heap_failure_lfh_bitmap_mismatch

PRIMARY_PROBLEM_CLASS: ACTIONABLE_HEAP_CORRUPTION_heap_failure_lfh_bitmap_mismatch

DEFAULT_BUCKET_ID: ACTIONABLE_HEAP_CORRUPTION_heap_failure_lfh_bitmap_mismatch

STACK_TEXT:
0225f520 77d924a1 c0000374 77dc0130 0225f564 ntdll!RtlReportCriticalFailure+0x33
0225f530 77d9168d 00000002 934d01f7 00000016 ntdll!RtlpReportHeapFailure+0x21
0225f564 77d6434b 0000000e 00ce0000 00dd22d0 ntdll!RtlpLogHeapFailure+0xa2
0225f604 77cc3561 00000050 00fa6728 00000000 ntdll!RtlpLowFragHeapAllocFromContext+0x2d4
0225f68c 680ca40f 00ce0000 00000000 00000050 ntdll!RtlAllocateHeap+0x105
0225f6d4 680c981a 00fa6728 00d4b774 00000000 component !DoMoreWork+0x37
0225f768 680c8f84 00000001 00d4b774 00000000 component !DoWork+0xce
0225f918 680d0169 00d27c48 02483530 02483590 component !Query+0x12b
0225f92c 77cf5935 00d4d170 00000000 02483530 component !WorkDispatchThreadProc+0x99
0225fa70 77cd9139 0225fad4 02483590 934d08cb ntdll!TppWorkpExecuteCallback+0x338
0225fc58 75fb2a32 00cef980 0225fca4 77d0cdfe ntdll!TppWorkerThread+0x6da
0225fc64 77d0cdfe 00cef980 934d0837 00000000 KERNEL32!BaseThreadInitThunk+0xe
0225fca4 77d0cdaa ffffffff 77d88566 00000000 ntdll!__RtlUserThreadStart+0x4a
0225fcb4 00000000 77d0d633 00cef980 00000000 ntdll!_RtlUserThreadStart+0x1c

...

1: kd> .frame 2
02 0225f564 77d6434b ntdll!RtlpLogHeapFailure+0xa2 [d:\5858\minkernel\ntos\rtl\heaplog.c @ 672]
1: kd> dv
0225f56c FailureType = heap_failure_lfh_bitmap_mismatch (0n14)
0225f570 HeapAddress = 0x00ce0000
0225f574 Address = 0x00dd22d0
0225f578 Param1 = 0x00000000
0225f57c Param2 = 0x00000000
0225f580 Param3 = 0x00000000

////////////////////////////////////////////////////////////////////////////////

My code in question is uninteresting:

pData =(PDATA)HeapAlloc(GetProcessHeap(),0, sizeof(DATA) *m_cData);

if(!pData) {

hr =E_OUTOFMEMORY;

goto Exit;

}

//////////////////////////////////////////////////////////////////////

So what is a HEAP_FAILURE_LFH_BITMAP_MISMATCH? I wasn't sure, so I had to find out. First off LFH refers to the Low-Fragmentation Heap which became the default in Windows Vista; you can read more about here. Basically the LFH uses bitmap masks to track whether blocks are free or busy. This information is also available in each LFH block's metadata. This failure indicates that the busy status between these two do not agree and is therefore corrupted. This still doesn't solve the mystery. Luckily there is a debugger extension help us find more clues, !heap.

/////////////////////////////////////////////////////////////

1: kd> !heap -triage ce0000
**************************************************************
* *
* HEAP ERROR DETECTED *
* *
**************************************************************

Details:

Heap address: 00ce0000
Error address: 00dd22d0
Error type: HEAP_FAILURE_LFH_BITMAP_MISMATCH
Details: The LFH detected a mismatch between an individual
block's metadata and its corresponding subsegment's
metadata.
Follow-up: Enable pageheap.
Error type: Unrecognized failure.
Follow-up: This may be a bug in the extension. Send a
remote or dump to ______.

Stack trace:
77d6434b: ntdll!RtlpLowFragHeapAllocFromContext+0x000002d4
77cc3561: ntdll!RtlAllocateHeap+0x00000105
680ca40f: component!DoMoreWork+0x00000037
680c981a: component!DoWork+0x000000ce
680c8f84: component!DoWork+0x0000012b
680d0169: Component!Query+0x00000099
77cf5935: ntdll!TppWorkpExecuteCallback+0x00000338
77cd9139: ntdll!TppWorkerThread+0x000006da
75fb2a32: KERNEL32!BaseThreadInitThunk+0x0000000e
77d0cdfe: ntdll!__RtlUserThreadStart+0x0000004a
77d0cdaa: ntdll!_RtlUserThreadStart+0x0000001c

**********************************************************
** !heap: Searching for the heap and segment that
** contain the specified address. To search
** for the entry that contains this address,
** use !heap -x 00ce0000.
**********************************************************

** !heap: Analyzing heap at 00ce0000...

** !heap: The following LFH allocations are missing a flag in their
'unused bytes' field that identifies them as LFH allocations.
This is usually caused by entry corruption in the client
application.
** !heap: To view the state of the invalid blocks:
!heap -i
!heap -i

Heap address Entry address Unused bytes
----------------------------------------------------------------------------
ce0000 dd1b98 49
ce0000 dd22d0 49
ce0000 dd2590 49
ce0000 dd1b98 49
ce0000 dd22d0 49
ce0000 dd2590 49
ce0000 dd1b98 49
ce0000 dd22d0 49
ce0000 dd2590 49
ce0000 dd1b98 49
ce0000 dd22d0 49
ce0000 dd2590 49

** !heap: If these failures are easily reproducible, they can
be detected as they occur by enabling pageheap for
this scenario.

1: kd>

///////////////////////////////////////////////////////////////////////////////////////

Ok, still no luck. I will try enabling pageheap if this issue is reproducible.

This is how you enable page heap verification.

//////// update /////////

The corruption of the heap is indicative of a buffer overrun. I think I have been able to pinpoint the source. I have a RTL linked list that tracks the state of sub-operations within a larger RPC client driven operation. It turned out there were two latent conditions where this list was not correctly locked: the first one was when a sub-operation failed to initialize and I would remove it out of the list without locking it, and the second was when sub-operations in other threads would send a state update (which causes the lock to be taken) at the same moment when the list was getting torn down. Obviously removing elements from the list while some other thread is actively traversing it can cause the other thread to be executing on bad Flinks. Fixing these issues should make this corruption go away. This was a very seldom repro on x86 and AMD64, but apparently common on ARM.

RTL linked lists

critical sections

Conditional WPP Tracing with WPP_LEVEL_ENABLED

2012-02-17T13:25:00.000-08:00

In general WPP tracing is a no-op if tracing is not enabled on your component. For instance, you don't need to worry about if this line is making your production code slower when tracing is not enabled:

LogTrace(MY_ERROR_LEVEL, L"Failed to start the RPC subsystem. hr = 0x%08x", hr);

But, there are times when you would like to trace something more involved than capturing an error and HRESULT.

For instance in this code, I wanted to be able to trace what SIDs I was generating, so to trace the string of the SID, some memory needed to be allocated. I didn't want this to happen in my production code if tracing wasn't enabled.

LPWSTR pszSid = NULL;

if (!ConvertSidToStringSid(pTokenGroups->Groups[i].Sid, &pszSid)) {
LogTrace(DAS_ERROR_LEVEL, "Could not generate SID string 0x%x\n", HRESULT_FROM_WIN32(GetLastError()));
}
else {
LogTrace(DAS_INFO_LEVEL, "Created process SID %ls\n", (LPCWSTR)pszSid);
}

if (pszSid) {
LocalFree(pszSid);
}

Sure, the LogTrace is a no-op if tracing isn't enabled, but I will always be allocating the string regardless. I didn't want that in my production code. This is where the WPP_LEVEL_ENABLED macro comes in.

WPP_LEVEL_ENABLED(level) can check to see if a certain level is currently enabled. It is generated by the preproccessor, and uses the same mechanism the other WPP generated functions use to decided when to actually do work.

So, to fix that code snip:

if (WPP_LEVEL_ENABLED(MY_INFO_LEVEL) || WPP_LEVEL_ENABLED(MY_ERROR_LEVEL)) {
LPWSTR pszSid = NULL;

if (!ConvertSidToStringSid(pTokenGroups->Groups[i].Sid, &pszSid)) {
LogTrace(MY_ERROR_LEVEL, "Could not generate SID string 0x%x\n", HRESULT_FROM_WIN32(GetLastError()));
}
else {
LogTrace(MY_INFO_LEVEL, "Created process SID %ls\n", (LPCWSTR)pszSid);
}

if (pszSid) {
LocalFree(pszSid);
}
}

Now ConvertSidToStringSid will not be called and the string will not be allocated unless the component's WPP info level or error level is enabled.

Use this macro then to programmatically skip work that is only needed for WPP logging when logging isn't turned on.

Looking at Heap Allocations With Windbg

2012-01-05T15:15:00.000-08:00

I will keep this brief. !heap is main debugger extension for allocation debugging. I will show you some tricks to get started using it. You can read up on all of the options by either typing !heap -? in the debugger or reading up on it in MSDN.

As always, don't do it the hard way. Use tools like Application Verifier to help you find issues more quickly. Appverif is awesome at identifying and even root causing many heap, leak, deadlock, and many other issues. You should get in the habit of always debugging with it on. Still, you should still know how to debug heap issues, so read on.

First think you will notice is that most of !heap's options take either a heap index or a heap address. You can use 0 for process' heap, but there can be a lot and you might have to wait a while. I will show you how to get it, and then you can play with the other options to learn what they do.

Generally you have a pointer of an allocation, and you want to know more about it. To find the heap for the allocation you are looking for there are two simple ways that I can think of.

!heap -x ptr_address

This will cause the debugger to search for the heap block with that address. This can take a little while depending on how many allocations there are. Using the -v option will take even longer, so you should probably avoid using it.

example, double free:
0:000> !analyze -v
...
STACK_TEXT:
008ffbf4 77b37d01 c0000374 77b67130 008ffc38 ntdll!RtlReportCriticalFailure+0x33
008ffc04 77b36eb5 00000002 91731eca 00c50000 ntdll!RtlpReportHeapFailure+0x21
008ffc38 77b06e39 00000008 00c50000 00c50d88 ntdll!RtlpLogHeapFailure+0xa2
008ffc6c 76de9898 00c50000 00000000 00c50d90 ntdll!RtlFreeHeap+0x5c
008ffcb8 00df14c2 00c50d90 00000000 00c50d90 msvcrt!free+0x65
008ffcd0 00df1884 00000001 00c50c80 00c52f10 program!wmain+0x42
008ffd20 00df193f 008ffd34 759c2914 7fe6f000 program!__wmainCRTStartup+0x164
008ffd28 759c2914 7fe6f000 008ffd74 77ab7fb6 program!wmainCRTStartup+0xf
008ffd34 77ab7fb6 7fe6f000 91731f86 00000000 KERNEL32!BaseThreadInitThunk+0xe
008ffd74 77ab7f62 ffffffff 77b2db43 00000000 ntdll!__RtlUserThreadStart+0x4a
008ffd84 00000000 00df1930 7fe6f000 00000000 ntdll!_RtlUserThreadStart+0x1c

FOLLOWUP_IP:
program!wmain+42 [s:\program\program.cpp @ 423]
00df14c2 83c404          add     esp,4

FAULTING_SOURCE_CODE:
   419:     UNREFERENCED_PARAMETER(argv);
   420:
   421:     p = malloc(1);
   422:     free(p);
> 423:     free(p);
   424:
   425:

So the double free in this case is obvious, but normally if you hit one it wasn't.

0:000> .frame 7
07 008ffcd0 00df1884 program!wmain+0x42 [s:\program\program.cpp @ 423]
0:000> dv
           argc = 1
           argv = 0x00c50c80
             hr = 0x00000000
              p = 0x00c50d90
0:000> !heap -x 0x00c50d90
Entry     User      Heap      Segment       Size PrevSize Unused    Flags
-----------------------------------------------------------------------------
00c50d88 00c50d90 00c50000 00c50000       4c0       110         0 free fill

0:000> !heap -a 00c50000
Index   Address Name      Debugging options enabled
3:   00c50000
    Segment at 00c50000 to 00c5f000 (00004000 bytes committed)
    Flags:                40001062
    ForceFlags:           40000060
    Granularity:          8 bytes
    Segment Reserve:      00100000
    Segment Commit:       00002000
    DeCommit Block Thres: 00000200
    DeCommit Total Thres: 00002000
    Total Free Size:      000002a7
    Max. Allocation Size: 7ffdefff
    Lock Variable at:     00c50248
    Next TagIndex:        0000
    Maximum TagIndex:     0000
    Tag Entries:          00000000
    PsuedoTag Entries:    00000000
    Virtual Alloc List:   00c5009c
    Uncommitted ranges:   00c5008c
            00c54000: 0000b000 (45056 bytes)
    FreeList[ 00 ] at 00c500c0: 00c52388 . 00c50d90
        00c50d88: 00110 . 004c0 [104] - free
        00c53af0: 00be8 . 004f0 [104] - free
        00c52380: 00818 . 00b88 [104] - free

Segment00 at 00c50000:
        Flags:           00000000
        Base:            00c50000
        First Entry:     00c50498
        Last Entry:      00c5f000
        Total Pages:     0000000f
        Total UnCommit: 0000000b
        Largest UnCommit:00000000
        UnCommitted Ranges: (1)

Heap entries for Segment00 in Heap 00c50000
         address: psize . size flags   state (requested size)
        00c50000: 00000 . 00498 [101] - busy (497)
        00c50498: 00498 . 00118 [107] - busy (117), tail fill Internal
        00c505b0: 00118 . 00230 [107] - busy (214), tail fill
        00c507e0: 00230 . 00498 [107] - busy (480), tail fill
        00c50c78: 00498 . 00110 [107] - busy (f8), tail fill
        00c50d88: 00110 . 004c0 [104] free fill        00c51248: 004c0 . 00238 [107] - busy (220), tail fill
        00c51480: 00238 . 00650 [107] - busy (632), tail fill
        00c51ad0: 00650 . 00098 [107] - busy (80), tail fill
        00c51b68: 00098 . 00818 [107] - busy (800), tail fill
        00c52380: 00818 . 00b88 [104] free fill
        00c52f08: 00b88 . 00be8 [107] - busy (bd0), tail fill
        00c53af0: 00be8 . 004f0 [104] free fill
        00c53fe0: 004f0 . 00020 [111] - busy (1d)
        00c54000:      0000b000      - uncommitted bytes.

You can see the memory is already freed.
If your heap call is on the stack at the time, you can find this value by walking the stack to and dumping variables. This can be helpful where there are a lot of allocations and -x will take a long time. The heap handle is the first parameter passed in to the ntdll RtlHeap*/Heap* functions.

example:
free((void*)1234);This will cause an access violation. 0:000> kn # ChildEBP RetAddr 00 0102fa6c 77b37d01 ntdll!RtlReportCriticalFailure+0x33 [d:\6588\minkernel\ntos\rtl\rtlutil.c @ 161] 01 0102fa7c 77b36eb5 ntdll!RtlpReportHeapFailure+0x21 [d:\6588\minkernel\ntos\rtl\heaplog.c @ 161] 02 0102fab0 77b06e4c ntdll!RtlpLogHeapFailure+0xa2 [d:\6588\minkernel\ntos\rtl\heaplog.c @ 672] 03 (Inline) -------- ntdll!RtlpProbeUserBufferUnsafe+0x8066d [d:\6588\minkernel\ntos\rtl\heappriv.h @ 2720] 04 (Inline) -------- ntdll!RtlpProbeUserBuffer+0x80677 [d:\6588\minkernel\ntos\rtl\heappriv.h @ 2754] 05 0102fae4 76de9898 ntdll!RtlFreeHeap+0x6c [d:\6588\minkernel\ntos\rtl\heap.c @ 1910] 06 0102fb30 00ce14a1 msvcrt!free+0x65 [d:\6588\minkernel\crts\crtw32\heap\free.c @ 183] 07 0102fb44 00ce1864 program!wmain+0x21 [s:\program\program.cpp @ 420] 08 0102fb94 00ce191f program!__wmainCRTStartup+0x164 [s:\dep\minkernel\crts\crtw32\dllstuff\crtexe.c @ 692] 09 0102fb9c 759c2914 program!wmainCRTStartup+0xf [s:\dep\minkernel\crts\crtw32\dllstuff\crtexe.c @ 510] 0a 0102fba8 77ab7fb6 KERNEL32!BaseThreadInitThunk+0xe [d:\6588\base\win32\client\thread.c @ 65] 0b 0102fbe8 77ab7f62 ntdll!__RtlUserThreadStart+0x4a [d:\6588\minkernel\ntdll\rtlstrt.c @ 1018] 0c 0102fbf8 00000000 ntdll!_RtlUserThreadStart+0x1c [d:\6588\minkernel\ntdll\rtlstrt.c @ 936] 0:000> .frame 5 05 0102fae4 76de9898 ntdll!RtlFreeHeap+0x6c [d:\6588\minkernel\ntos\rtl\heap.c @ 1910] 0:000> dv HeapHandle = 0x010d0000 Flags = 0 BaseAddress = 0x000004d2 BusyBlock = 0x00000000 Interceptor = 0:000> !heap -a 0x010d0000Index Address Name Debugging options enabled 3: 010d0000 Segment at 010d0000 to 010df000 (00004000 bytes committed) Flags: 40001062 ForceFlags: 40000060 Granularity: 8 bytes Segment Reserve: 00100000 Segment Commit: 00002000 DeCommit Block Thres: 00000200 DeCommit Total Thres: 00002000 Total Free Size: 000002a7 Max. Allocation Size: 7ffdefff Lock Variable at: 010d0248 Next TagIndex: 0000 Maximum TagIndex: 0000 Tag Entries: 00000000 PsuedoTag Entries: 00000000 Virtual Alloc List: 010d009c Uncommitted ranges: 010d008c 010d4000: 0000b000 (45056 bytes) FreeList[ 00 ] at 010d00c0: 010d2388 . 010d0d90 010d0d88: 00110 . 004c0 [104] - free 010d3af0: 00be8 . 004f0 [104] - free 010d2380: 00818 . 00b88 [104] - freeSegment00 at 010d0000: Flags: 00000000 Base: 010d0000 First Entry: 010d0498 Last Entry: 010df000 Total Pages: 0000000f Total UnCommit: 0000000b Largest UnCommit:00000000 UnCommitted Ranges: (1)Heap entries for Segment00 in Heap 010d0000 address: psize . size flags state (requested size) 010d0000: 00000 . 00498 [101] - busy (497) 010d0498: 00498 . 00118 [107] - busy (117), tail fill Internal 010d05b0: 00118 . 00230 [107] - busy (214), tail fill 010d07e0: 00230 . 00498 [107] - busy (480), tail fill 010d0c78: 00498 . 00110 [107] - busy (f8), tail fill 010d0d88: 00110 . 004c0 [104] free fill 010d1248: 004c0 . 00238 [107] - busy (220), tail fill 010d1480: 00238 . 00650 [107] - busy (632), tail fill 010d1ad0: 00650 . 00098 [107] - busy (80), tail fill 010d1b68: 00098 . 00818 [107] - busy (800), tail fill 010d2380: 00818 . 00b88 [104] free fill 010d2f08: 00b88 . 00be8 [107] - busy (bd0), tail fill 010d3af0: 00be8 . 004f0 [104] free fill 010d3fe0: 004f0 . 00020 [111] - busy (1d) 010d4000: 0000b000 - uncommitted bytes.

Retrieving GetLastError() in Windbg

2011-12-06T17:28:00.001-08:00

Let's say that you are debugging some code with windbg and some error happened, but the code doesn't call GetLastError() and store it in a local variable. Don't worry, you can still find out what the error was. It is stored in the TEB (thread environment block) and there is a debugger extension for the TEB.

0:002> !teb
TEB at 7f38d000
    ExceptionList:        018bfad0
    StackBase:            018c0000
    StackLimit:           018bc000
    SubSystemTib:         00000000
    FiberData:            00001e00
    ArbitraryUserPointer: 00000000
    Self:                 7f38d000
    EnvironmentPointer:   00000000
    ClientId:             0000076c . 00000518
    RpcHandle:            00000000
    Tls Storage:          0115a170
    PEB Address:          7f384000
    LastErrorValue:       1168
    LastStatusValue:      c000000d
    Count Owned Locks:    0
    HardErrorMode:        0
0:002> !error c000000d
Error code: (NTSTATUS) 0xc000000d (3221225485) - An invalid parameter was passed to a service or function.
0:002> !error 0n1168
Error code: (Win32) 0x490 (1168) - Element not found.

Note LastErrorValue is in decimal and not hex so be sure to add the 0n when using !error. There you go, there is the last error.

source

new nothrow

2011-12-02T11:38:00.001-08:00

I don't use STL, and I don't C++ exceptions. I avoid them for a few reasons: under debugger, they make it a pain to find and walk the faulting call stack after they are thrown, and mostly I lean towards a more C style of programming style personally. Still, you need to use the new operator to create new C++ objects, and new can throw and exception in low resource conditions. There are a few ways to change this default behavior so that it just returns NULL instead of throwing.

One way is to add this into your sources file:


TARGETLIBS=\    
    $(SDK_LIB_PATH)\nothrownew.obj       \

Another way is to add a no throw each time you use the new operator:


    ClassX *pObj = new (std::nothrow) ClassX();

    if (!pObj)
        return E_OUTOFMEMORY;

Synchronization Primitives Available in Win32

2011-11-15T19:14:00.001-08:00

I ran into this table in a concurrency training class. It gives you the quick and dirty on synchronization primitives available in Win32.

Characteristic	CriticalSection	SRWLocks	Mutex Semaphores Events
Kernel call even if free and upon creation	N	N	Y
Non-FIFO (no convoys)	Y	Y	N
Spin before wait option	Y	Y	N
Try to acquire option	Y	N/Y on Win7	N
Cross-process	N	N	Y
Recursive acquires	Y	N	Mutex only
Use in WaitForMultiple..	N	N	Y

CriticalSections are the lightest weight and normally the best to use.

you can use them recursively on the same thread

SRWLocks are best if you need to differentiate reading and writing access. For example you can take a shared lock for read only access to a shared resource and exclusive lock for writing.

It is generally better than just using a critical section if the ratio is 2 reads : 1 write
Cannot be used recursively

Mutex/Semaphores/Events are kernel objects

Can be used across process.
Obviously available in krenel mode.
In user mode, the will be less performant than critical sections and SRWlocks and will cause more context switching.
Are processed in convoys so they can avoid starvation

Instrumentation is the best way to know what kind of synchronization is best for your code. Look at my previous post on how to instrument with ETW and XPerf for industrial grade perf testing. Try different synchronization mechanisms and measure performance.

How to Add Instrumentation to Your Manifest

2011-08-04T15:05:00.000-07:00


To enable ETW performance tracing for your binary, the first thing you need to do is add an instrumentation section.  Here is the MSDN documentation on how to do this if you want to read about this in detail:
   
  http://msdn.microsoft.com/en-us/library/aa382776(v=VS.85).aspx
   
  Also in the introduction for ETW (http://msdn.microsoft.com/en-us/magazine/cc163437.aspx), they make a tool recommendation for generating a manifest for ETW (ecmangen.exe).  It is available in the Windows SDK.  You can try it out; it may meet your needs just fine.  As a person who still favors writing code in Vi, I opted to roll my own.  Tools are great when they work for you.  Using a tool; however, doesn’t make up for actually knowing how things work.  It is good to understand how things work when things don’t come together.  Read on if you would rather roll your own.

  So below is the events section instrumentation manifest that has one start and stop even.  You will need a start and stop for everything you want to time.
   
          <events>
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5501)"
                opcode="win:Start"
                symbol="CREATE_QUERY_START"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5500"
                />
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5502)"
                opcode="win:Stop"
                symbol="CREATE_QUERY_STOP"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5501"
                />
          events>
   
  With some highlighting, I will show you where all the bits line up inside the full manifest.
   
          <events>
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5501)"
                opcode="win:Start"
                symbol="CREATE_QUERY_START"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5500"
                />
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5502)"
                opcode="win:Stop"
                symbol="CREATE_QUERY_STOP"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5501"
                />
          events>
   
  Here is the full manifest:
   
  xml version='1.0' encoding='utf-8' standalone='yes'?>
  <assembly
      xmlns="urn:schemas-microsoft-com:asm.v3"
      xmlns:xsd="http://www.w3.org/2001/XMLSchema"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      manifestVersion="1.0"
      >
    <assemblyIdentity
        buildType="$(build.buildType)"
        language="neutral"
        name="Microsoft-Windows-Component"
        processorArchitecture="$(build.arch)"
        publicKeyToken="$(Build.WindowsPublicKeyToken)"
        version="$(build.version)"
        versionScope="nonSxS"
        />
    <instrumentation xmlns:win="http://manifests.microsoft.com/win/2004/08/windows/events">
      <events xmlns="http://schemas.microsoft.com/win/2004/08/events">
        <provider
            guid="{56c71c31-cfbd-4cdd-8559-505e042bbbe1}"
            message="$(string.EventProviderName)"
            messageFileName="%SystemRoot%\system32\component.dll"
            name="Microsoft-Windows-ComponentProvider"
            resourceFileName="%SystemRoot%\system32\component.dll"
            symbol="component"
            >
          <channels>
            <channel
                chid="50"
                name="Microsoft-Windows-ComponentProvider/Performance"
                symbol="MS_COMPONENT_PERFORMANCE"
                type="Analytic"
                />
          channels>
          <keywords>
            <keyword
                mask="0x1"
                name="PerfTrackContext"
                />
            <keyword
                mask="0x2"
                name="Component"
                />
            <keyword
                mask="0x4"
                name="query"
                />
          keywords>
          <tasks>
            <task
                name="component_CreateQuery"
                value="5000"
                />
          tasks>
          <templates>
            <template tid="INFO_CONTEXT_INFO">
              <data
                  inType="win:Pointer"
                  name="Context"
                  />
            template>
          templates>
          <events>
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5501)"
                opcode="win:Start"
                symbol="CREATE_QUERY_START"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5500"
                />
            <event
                channel="50"
                keywords="Component query"
                level="win:Informational"
                message="$(string.event_5502)"
                opcode="win:Stop"
                symbol="CREATE_QUERY_STOP"
                task="component_CreateQuery"
                template="INFO_CONTEXT_INFO"
                value="5501"
                />
          events>
        provider>
      events>
    instrumentation>
    <localization>
      <resources culture="en-US">
        <stringTable>
          <string
              id="EventProviderName"
              value="Microsoft-Windows-Component"
              />
          <string
              id="event_5501"
              value="create query has started."
              />
          <string
              id="event_5502"
              value="create query has exited."
              />
        stringTable>
      resources>
    localization>
    <dependency
        discoverable="false"
        optional="false"
        resourceType="Resources"
        >
      <dependentAssembly dependencyType="prerequisite">
        <assemblyIdentity
            buildType="$(build.buildType)"
            language="*"
            name="Microsoft-Windows-Component.Resources"
            processorArchitecture="$(build.processorArchitecture)"
            publicKeyToken="$(Build.WindowsPublicKeyToken)"
            version="$(build.version)"
            versionScope="nonSxS"
            />
      dependentAssembly>
    dependency>
  assembly>

Performance Instrumentation Testing for Windows Programming

2011-07-18T21:00:00.000-07:00

Performance is something you should always build in as you are writing code whether you are building something from the ground up or working on bug fixes for some legacy code. Performance is one of those core software principals that is just good hygiene. Similarly you should always write secure code, reliable code, et cetera, but those are separate topics.

Performance considerations start at the very beginning when you start designing your features, and continue throughout the process. So, let’s say that you already came up with a high performance design that you validated through prototyping. Great, you are starting off the project on the right foot; as you know it becomes much harder to make design changes as the project goes down the pike, or even impossible towards the end of the development cycle.

So now you are in the thick of coding, or perhaps on the tail end of development, and you want to make sure that you are still meeting your performance goals for your feature work. Saying you are a good programmer and you always right high performance code isn’t quite good enough; you have to actually test your code. Writing code with timers in it and doing a printf with times is ok for prototypes, but doesn’t work for real production code. Not to worry, there are a number of tools at the Windows developer’s disposal to actually do the job right.

Profiling is one tool in the toolbox; one such profiler is Microsoft’s F1 made by the developer division. I think it is available in the wild as a standalone, or it looks like it has been included with VS08 and VS10. I personally have used the standalone F1 and still write code in Vim, but maybe I am weird like that. The basic idea of F1 is that it breaks into your process at a certain time interval and check to see what is on your threads’ call stacks. Based on what is on the stacks, F1 can give you ideas as to how much time is spent in each function, and about allocations being made. The advantages to using a profiler like F1 are: you don’t have to modify your binaries to instrument, it is light weight, and it gives you an idea how much time your code spends in functions. Some downsides are: you don’t get call count information, CPU usage is skewed by timer caused context switching, doesn’t profile kernel mode usage, it is less effective for system wide issues, it is heavy weight, and it doesn’t scale well if you want to profile often. To sum it up, F1 is a great tool with limitations. It really can give you a real-world performance view of your code (if it runs in UM), but there are better ways to instrument if want to look at performance more than a few times.

One of these better ways is ETW (Event Tracing for Windows). I will not get into the nitty-gritty of ETW; I will assume that the reader knows how to Google and can read the copious content out there, but I will give you a quick a dirty idea as to what it is. Basically ETW is the Windows operating system’s built in tracing mechanism. It is baked in all the way to the kernel level, and is very high performance. Since ETW is baked into the operating system, you can track scenarios from process to process to even KM components with very little overhead. You can use it for many other things as well, like writing events to logs and so on, but I will focus on using ETW for instrumenting your code for performance. For some reason though, I haven’t found too much help on the Internet on how to do performance instrumentation. I plan to help with that.

In the days and weeks to come, I will break it down how to instrument your code with ETW, and use the Windows tools for performance tracking and testing.

ETW Overview: http://msdn.microsoft.com/en-us/magazine/cc163437.aspx

Do not confuse ETW with WPP. Yes WPP is built on ETW, but I use WPP for more conventional printf style tracing. It is good to also add WPP tracing to your code for debugging purposes, but I don’t use it for performance testing. In fact WPP tracing is probably the best way to debug nasty concurrency issues that don’t repro well on a debugger, so you should look into it.

CreateThread Usage Pattern

2011-01-07T15:27:00.000-08:00

I don’t really use the old thread pool (Windows 2000 & XP) in favor of the new awesome thread pool introduced in Vista, but sometimes you might inherit old code that does. I just finished fixing a bug related to the old CreateThread. I have some time before my team’s Friday afternoon party, so I thought I would share. I found the error thanks to App Verifier; +1 to App Verifier’s awesomeness.

Basically the usage pattern is:
1. get the module handle (GetModuleHandleEx)
2. create the thread (CreateThread)
3. If COM, CoInitialize a new apartment for the thread
4. Do work
5. If COM, CoUninitialize
6. FreeLibraryAndExitThread with the handle gotten in #1
7. Return

Steps 1-2, The thread creator would do something like this:


if (!GetModuleHandleEx(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS, (LPCWSTR)&Class::ThreadProc , &s_hModule))
    hr = HRESULT_FROM_WIN32( GetLastError() ); 

if (S_OK == hr) {
    if (s_hThread)
        CloseHandle(s_hThread);
    s_hThread = CreateThread(NULL, 0, ThreadProc, this, 0, &s_dwThreadID);
    if (!s_hThread) {
        hr = HRESULT_FROM_WIN32(::GetLastError());
        FreeLibrary( s_hModule );
    }
}

In my case, the bug was in the GetModuleHandleEx. Take heed that if you are using the GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRES flag, then the second parameter should an address inside of your DLL; I have read that &__ImageBase or the HINSTANCE passed into DllMain should also do the trick. Also the third parameter will be used inside the thread proc.

Steps 3-7:


DWORD WINAPI Class::ThreadProc(void *pv)
{
    if (S_OK == CoInitializeEx( NULL, COINIT_MULTITHREADED)) {
        //...
        // do work here
        CoUninitialize();
    }

    FreeLibraryAndExitThread(s_hModule, 0);
    return 0;
}

As always, read the documentation to go deeper in the topic.

Setting a Processor Breakpoint in WinDbg

2011-01-07T10:35:00.001-08:00

If you have a variable like a pointer or a ref count that seems to be getting clobbered or incorrectly accessed, the ba (break on access) command can be a life saver. Setting a break on the memory address allows you to break in with windbg, inspect the stack, or do whatever else you need to do the moment the memory is accessed.

The basic usage is:
ba address access memory_size

Ex. Tracking write access of a pointer on an AMD64 machine:
ba 0000000070540018 w8

Ex. Tracking read/write of a DWORD ref count on x86:
ba 7124220 r4

Look at the windbg documentation for info.

HOWTO Debug a Windows Service Using windbg

2010-08-03T18:02:00.001-07:00

There are a lot of ways to debug a Windows service.

The simplest is to attach the debugger directly.
1. Use Task Manager to discover the PID for the service or service host. You can also use tlist.exe.
2. Use windbg and attach to that PID

Oftentimes you need to debug the service right from startup. Also it is a good idea to isolate the service in its own process since Windows generally runs several processes that have the same security context in the same host process.
1. Stop the service - 'net stop service_name'
2. Edit the following keys in the registry:
- HKLM\System\CurrentControlSet\Control - create new DWORD value - name = 'ServicesPipeTimeout', value = 86400000. The longer timeout gives you a chance to attach a debugger and set things up.
- HKLM\System\CurrentControlSet\Services\service_name - edit the ImagePath to launch myhost.exe instead of svchost.exe
- HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options - Add a new subkey called 'myhost.exe'. Under it, create a 'String' value called 'Debugger' and set the value to "C:\debuggers\ntsd.exe -server npipe:pipe=service_name_debug"
3. Make a copy of svchost.exe and name it myhost.exe ('copy \windows\system32\svchost.exe myhost.exe')
4. Enable the service to run in its own process - 'sc config service_name type= own'
5. You might need to reboot now if the service dll was in use still (was not unloaded on service stop). Also, it's better to mark the service as 'Manual' start so that it doesn't start automatically at boot - this way you can be ready when you launch the service.
6. Start the service – ‘net start service_name’
7. Attach the debugger ‘windbg –remote npipe:pipe=service_name_debug,server=localhost’. In the debugger, you can setup breakpoints, exceptions, and whatever else you need to do to debug the service, then type ‘g’ to let the service startup.

This article provides more ways to debug a Windows service.
http://support.microsoft.com/kb/824344

Link Error LNK2001 __cdecl

2010-03-09T09:03:00.000-08:00

You may have ran into a link error like this while trying to link your driver or something.

Linking Executable - s:\depot.obj.amd64\proj\objchk\amd64\proj.dll
errors in directory s:\depot\proj
s:\depot\proj\proj.obj : error LNK2001: unresolved external symbol "long __cdecl _LibFunction(unsigned long,void *, ...)" (?_LibFunction@@YAJKPEAX000PEAPEAUHPNPCTX__@@@Z)

This recently happened to me while I was trying to link a C lib file into my C++ project. If this doesn't already cue you to the problem, this link error is caused by trying to link a C lib object in my C++ code. In the lib's header or around the #include in your C++ project, wrap the prototypes with this:

#if __cplusplus
extern "C" {
#endif

_LibFunction(...);
...

#if __cplusplus
}
#endif

If you just forgot to add the lib to your TARGETLIBS macro in your sources file, the error would look like this instead:

Linking Executable - s:\depot.obj.amd64chk\proj\objchk\amd64\proj.dll
errors in directory s:\depot\proj
s:\depot\proj.obj : error LNK2001: unresolved external symbol _LibFunction

Automatically Launching the Debugger windbg

2010-02-01T16:15:00.000-08:00

Having the debugger attach to a process as soon as it is created is something that I always end up needing to do, but don’t do every day and sometimes forget how to do it. Today I needed to do it again, so one of my coworkers reminded me how to do it.

One way is to just debug with the kernel debugger which is the more hardcore way to always debug. Once you have attached remotely to your test machine with the kernel debugger, you can go to any process you want. I think this is the best way if you regularly write kernel code.

I don’t write kernel code all that often, so I normally use windbg on the machine directly. If you need to debug two processes because the talk to each other via DCOM or RPC directly, you will need to attach windbg to each process. If the other process launches on demand, you will need a way to automatically launch windbg to attach to the process. There is a mechanism built into Windows for just the thing. You can read this MSDN blog post about it.

http://blogs.msdn.com/junfeng/archive/2004/04/28/121871.aspx

The short of it is to add the value “Debugger” to a key that is the name of your executable at:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options

So, the common case would be to add the key “dllhost.exe” and then give that key a new string value called “Debugger” with “windbg.exe” as the data.

BEGIN_OBJECT_MAP Removal Update

2010-01-26T11:34:00.000-08:00

I recently run into a bug in where my dll was supposed to support multiple objects. In my previous post, I was just supporting one. When com was calling DllGetClassObject, it was always just getting one object, which was incorrect. So this is how I went about fixing it.
Imagine you ATL code had something like this:


BEGIN_OBJECT_MAP(ObjectMap)
   OBJECT_ENTRY(CLSID_Obj1, CObj1)
   OBJECT_ENTRY(CLSID_Obj2, CObj2)
   OBJECT_ENTRY(CLSID_Obj3, CObj3)
END_OBJECT_MAP()

In your class factory, you need to be able to create the correct object. First I fixed my class factory object.
I added a new member variable:
REFCLSID m_clsid;
And got is set in the constructor:


CClassFactory::CClassFactory(
   REFCLSID clsid) : m_cRef(1), m_clsid(clsid)
{
   IncModuleCount();
}  // CClassFactory::CClassFactory

Next I fixed my CreateInstance method


STDMETHODIMP CClassFactory::CreateInstance(
   __in_opt IUnknown *pUnkownOuter,
   REFIID riid,
   __deref_out_opt void **ppv)
{
   HRESULT   hr = S_OK;
   IUnknown *pUnknown = NULL;

   if (ppv)
   {
       *ppv = NULL;
   }
   else
   {
       hr = E_INVALIDARG;
   }

   if (S_OK == hr)
   {
       if (pUnkownOuter)
       {
           hr =  CLASS_E_NOAGGREGATION;
       }
   }

   if (S_OK == hr)
   {
       if (CLSID_Obj1 == m_clsid)
       {
           pUnknown = new CObj1();
       }
       else if (CLSID_Obj2 == m_clsid)
       {
           pUnknown = new CObj2();
       }
       else if (CLSID_Obj3 == m_clsid)
       {
           pUnknown = new CObj3();
       }
       else
       {
           hr = E_NOINTERFACE;
       }

       if (S_OK == hr && !pUnknown)
       {
           hr =  E_OUTOFMEMORY;
       }
   }

   if (S_OK == hr)
   {
       hr = pUnknown->QueryInterface(riid, ppv);
   }

   if (pUnknown)
   {
       pUnknown->Release();
   }

   return hr;
}  // CClassFactory::CreateInstance

And then fixed my DllGetClassObject


STDAPI  DllGetClassObject(
   __in REFCLSID rclsid,
   __in REFIID riid,
   __deref_out LPVOID FAR *ppv)
{
   HRESULT hr = S_OK;
   CClassFactory *pClassFactory = NULL;

   if (ppv)
   {
       *ppv = NULL;
   }
   else
   {
       hr = E_INVALIDARG;
   }

   if (S_OK == hr)
   {
       pClassFactory = new CClassFactory(rclsid);

       if (!pClassFactory)
       {
           hr =  E_OUTOFMEMORY;
       }
   }

   if (S_OK == hr)
   {
       hr = pClassFactory->QueryInterface(riid, ppv);
   }

   if (pClassFactory)
   {
       pClassFactory->Release();
   }

   return hr;
}

Windows Win32 Timer or Time Counting

2009-12-14T13:38:00.000-08:00

If you want to make a timer that only depends on the kernel, GetTickCount64 is an easy way to do it.

http://msdn.microsoft.com/en-us/library/ms724411%28VS.85%29.aspx

The code is simple and would look like this:


ULONGLONG Ticks = 0;

Ticks = GetTickCount64();

//
// some code to time
//

Ticks = GetTickCount64() - Ticks;

Note: GetTickCount64 returns the number of ms since the system was last booted. Theoretically you might have check for wrapping of the clock; however, 2^64 ms or ~49.7 days is a long time. If you are worried about that, you might consider other APIs that keep track of the full time with a ms resolution.

Running cmd.exe as SYSTEM on Win 7 and Vista

2009-12-11T18:06:00.001-08:00

In Windows, being Administrator has its limitations. Running as SYSTEM is all powerful. It is kind of like being root in the UNIX world except you can just su or sudo to SYSTEM.

I have this lib that can run in kernel mode and in user mode. I would like to simulate a UM process using the lib doing the same things I would have it do in KM tp measure the performance difference. To solve this issue, I need to run the process as SYSTEM.

A while back, I remember doing exactly that in a sysintenrals class. I ran across this blog post that reminded me how to do it.

1. Download the sysinternals PSTools
2. Copy the files to somewhere in your path
3. In an admin cmd.exe run > psexec -i -s cmd.exe

That is it. Anything you run in the shell will be run as LOCAL SYSTEM.

ATL Removal: Part 6 – Replacing CComQIPtr with _com_ptr_t

2009-10-27T16:50:00.000-07:00

ATL Removal: Part 6 – Replacing CComQIPtr with _com_ptr_t
In part 4 of ATL removal, I talked about removing CComPtr and replacing it with _com_ptr_t. You may run into some smart pointers in the form of CComQIPtr. CComPtr and CComQIPtr have the same base class and CComQIPtr is a super set of CComPtr. I tried to read up on CComQIPtr in MSDN, but the documentation wasn’t really very helpful. In the code I am working on, CComQIPtr is used to create a smart pointer and then do a QueryInterface hence the QI. I verified this is what goes on by looking through the atlbase.h header file from the SDK.

If you don’t want to depend on ATL in the sense that your binary doesn't load the ATL dlls, the it is fine to still include the .h file, but if you want to purge ATL altogether, read on. Keep in mind _com_ptr_t doesn't do everything that ATL smart pointers, so some extra manual work may still be required. This is an example of a case where the extra legwork is required.

CComQIPtr(IUnknown* lp)
{
        p=NULL;
        if (lp != NULL)
                lp->QueryInterface(*piid, (void **)&p);
}

Before I had some code that looked like this:

CComQIPtr spLocaclVariable( m_spMemberProperty);

To fix it, I changed it to this:

_com_ptr_t<_com_IIID< IYourInterface, &__uuidof(IYourInterface)> > spLocaclVariable;
if (spMemberProperty)
{
    spMemberProperty ->QueryInterface(__uuidof(IYourInterface), (void**)& spLocaclVariable);
}

I think this should do the trick for your CComQIPtr smart pointers.

ATL Removal: Part 5 – Changing ATL’s Lock and Unlock to Slim Reader/Writer (SRW) Locks

2009-10-27T13:39:00.000-07:00

You may encounter Lock() and Unlock() calls in your ATL based code to synchronize the usage of shared memory amongst your threads. You get these bonus methods thanks to inheriting from ATL base classes (CComObjectRoot). Since we are removing ATL, we will have to find an alternative. Enter Slim Reader/Writer (SRW) locks.

SRW locks are not loaded with tons of features like some of the other locks that Microsoft has to offer ; however, they fast and easy on the memory foot print. In fact, you can consider using SRW locks instead of ATL locking as a performance enhancement. I won’t get into SRW locks here, but you can read more about them on MSDN here:

http://msdn.microsoft.com/en-us/library/aa904937%28VS.85%29.aspx

The first thing to do is add a SRWLOCK member like so:

SRWLOCK m_Lock;

Then in your class’ constructor add the initialization:

InitializeSRWLock(&m_Lock);

Replace Lock() with this:

AcquireSRWLockExclusive(&m_Lock);

And Unlock() with this:

ReleaseSRWLockExclusive(&m_Lock);

As you can see from the MSDN documentation SRW locks can do more than exclusive locks. You might want to look at these other SRW functions to give your critical sections just the right amount of protection.

The following are the SRW lock functions (from MSDN).

SRW lock function	Description
AcquireSRWLockExclusive	Acquires an SRW lock in exclusive mode.
AcquireSRWLockShared	Acquires an SRW lock in shared mode.
InitializeSRWLock	Initialize an SRW lock.
ReleaseSRWLockExclusive	Releases an SRW lock that was opened in exclusive mode.
ReleaseSRWLockShared	Releases an SRW lock that was opened in shared mode.
SleepConditionVariableSRW	Sleeps on the specified condition variable and releases the specified lock as an atomic operation.

ATL Removal: Part 4 – Replacing CComPtr with _com_ptr_t

2009-10-26T18:19:00.000-07:00

CComPtr (a.k.a. smart pointer) is another helpful template class provided in ATL that is intended to make COM programming easier. CComPtr wraps and abstracts the com object and does all of the lifetime management stuff automatically for you like calling AddRef and Release. Lifetime management is not that tricky to do it on your own. A pattern I like to use is call AddRef when I get passed a com object that I need to use, and then call Release when I am done using it. It’s not very complicated to get it right. Also, just because you are using a CComPtr, it is possible to still screw up the ref count and cause leaks or double frees, so you still need to know what you are doing with them to avoid these issues. Let’s move on to the topic at hand, replacing CComPtr.

Trying to just get rid of CComPtrs and code straight com objects could be a pain since it more about redoing the entire coding pattern of the project instead of modifying declarations. If you don’t want to depend on ATL in the sense that your binary doesn't load the ATL dlls, the it is fine to still include the .h file, but if you want to purge ATL altogether, read on. Keep in mind _com_ptr_t doesn't do everything that ATL smart pointers, so some extra manual work may still be required.

Luckily there is compiler support for non-ATL smart pointers in the form of _com_ptr_t that will allow us to change the declarations instead of the coding pattern.

http://msdn.microsoft.com/en-us/library/417w8b3b(VS.80).aspx

So this is what a declaration might look like before:

CComPtr m_spYourInterface;

And this is what it would look like afterwards:

_com_ptr_t<_com_iiid<> > m_spYourInterface;

Additionally you will need to add comip.h to your stdafx.h file, and USE_MSVCRT=1 comsuppw.lib to your TARGETLIBS in the sources file to get your project to compile and link.

Here are some errors you might hit:

file.cpp(776) : error C2039: 'CopyTo' : is not a member of '_com_ptr_t<_iiid>'

The code looks something like this:

hr = m_spObject.CopyTo( ppObject );

Change it to this:

if (m_spObject) m_spObject.AddRef();
*ppObject = m_spObject;

I guess _com_ptr_t<_iiid> does not provide that method, but the little AddRef and assignment does the same thing.

You might see this linking error as well:
error LNK2001: unresolved external symbol "void __stdcall _com_issue_error(long)" (?
_com_issue_error@@YGXJ@Z)

You need to make sure you enable compiler support for _com_ptr_t smart pointers. Add the following to your sources file:

USE_MSVCRT=1

ATL Removal: Part 3 – Tackling _Module and OBJECT_MAP

2009-10-23T13:38:00.000-07:00

In Part 1, we saw how ATL uses com maps to automatically generate the IUnknown implementation and to manage the objects lifetime. In this installment we will tackle the object and lifetime management of the DLL.

Let’s dive into the guts of the DLL management. This is another place that ATL was intended to save time for the developer. If you have an ATL based DLL, you might notice something like this in your main DLL source file.

CComModule _Module;
BEGIN_OBJECT_MAP(ObjectMap)
  OBJECT_ENTRY(CLSID_YourClass, CYourClass)
END_OBJECT_MAP()

The _Module object is designed to manage the lifetime of your DLL. You ATL CComObjects objects will automatically call _Module.Lock() and _Module.Unlock(). Awsome! If you use _Module in your ATL based DLL and some non ATL com objects, make sure the lock and unlock _Module in the constructors and destructors respectively; I fixed a bug last year where my DLL was prematurely getting unloaded because of this. The first thing to do to replace _Module is to add:

LONG g_cLockCount = 0;

inline VOID IncModuleCount()
{
  InterlockedIncrement(&g_cLockCount);
}  // IncModuleCount

inline VOID DecModuleCount()
{
  InterlockedDecrement(&g_cLockCount);
}  // DecModuleCount

You DLL can use that to keep track of the active objects so I can know when it is safe to unload.

Next let’s look at the object map. What does that buy for you? It basically provides you a free implementation of IClassFactory. Is it hard to write your own? No, I will show you how right now.
You can create a file called ClassFactory.h that looks like this:

class CClassFactory:
  public IClassFactory
{
public:
  // IUnknown
  STDMETHODIMP_(ULONG) AddRef();
  STDMETHODIMP_(ULONG) Release();
  STDMETHODIMP QueryInterface(
      REFIID riid,
      __deref_out_opt void **ppv);

  // IClassFactory
  STDMETHODIMP CreateInstance(
      __in_opt IUnknown *punkOuter,
      REFIID iid,
      __deref_out_opt void **ppv);

  STDMETHODIMP LockServer(
      BOOL fLock);

  // Constructor / Destuctor
  CClassFactory();
  ~CClassFactory();

protected:
  LONG m_cRef;
}; // CClassFactory

Now, let’s look at the implementation. Here is the content of ClassFactory.cpp:

#include "stdafx.h"

//---------------------------------------------------------------------------
// Begin CClassFactory implemetation
//---------------------------------------------------------------------------
extern LONG g_cLockCount;

CClassFactory::CClassFactory():
  m_cRef(1)
{
  InterlockedIncrement(&g_cLockCount);
}  // CClassFactory::CClassFactory

CClassFactory::~CClassFactory()
{
  InterlockedDecrement(&g_cLockCount);
}  // CClassFactory::~CClassFactory

STDMETHODIMP_(ULONG) CClassFactory::AddRef()
{
  return InterlockedIncrement(&m_cRef);
}  // CClassFactory::AddRef

STDMETHODIMP_(ULONG) CClassFactory::Release()
{
  LONG cRef = InterlockedDecrement(&m_cRef);

  if (!cRef)
  {
      delete this;
  }

  return cRef;
}  // CClassFactory::Release

STDMETHODIMP CClassFactory::QueryInterface(
  REFIID riid,
  __deref_out_opt void **ppv)
{
  HRESULT hr = S_OK;

  if (ppv)
  {
      *ppv = NULL;      
  }
  else
  {
      hr = E_INVALIDARG;
  }

  if (S_OK == hr)
  {
      if (IID_IUnknown == riid)
      {
          AddRef();
          *ppv = (IUnknown*)(IClassFactory*)this;
      }
      else if (IID_IClassFactory == riid)
      {
          AddRef();
          *ppv = (IClassFactory*)this;
      }
      else
      {
          hr = E_NOINTERFACE;
      }
  }

  return hr;
}  // CClassFactory::QueryInterface

STDMETHODIMP CClassFactory::CreateInstance(
  __in_opt IUnknown *pUnkownOuter,
  REFIID riid,
  __deref_out_opt void **ppv)
{
  HRESULT   hr = S_OK;
  IUnknown *pUnknown = NULL;

  if (ppv)
  {
      *ppv = NULL;
  }
  else
  {
      hr = E_INVALIDARG;
  }

  if (S_OK == hr)
  {
      if (pUnkownOuter)
      {
          hr =  CLASS_E_NOAGGREGATION;
      }
  }

  if (S_OK == hr)
  {
      pUnknown = new(std::nothrow) CAudioProvider();

      if (!pUnknown)
      {
          hr =  E_OUTOFMEMORY;
      }
  }

  if (S_OK == hr)
  {
      hr = pUnknown->QueryInterface(riid, ppv);
  }

  if (pUnknown)
  {
      pUnknown->Release();
  }

  return hr;
}  // CClassFactory::CreateInstance

STDMETHODIMP CClassFactory::LockServer(
  BOOL fLock)
{
  if (fLock)
  {
      InterlockedIncrement(&g_cLockCount);
  }
  else
  {
      InterlockedDecrement(&g_cLockCount);
  }

  return S_OK;
}  // CClassFactory::LockServer
//---------------------------------------------------------------------------
// End CClassFactory implemetation
//---------------------------------------------------------------------------

Next you will have to fix the rest of the DllMain cpp file. This is what a simple version would look like clean of ATL

extern "C"
{

BOOL APIENTRY DllMain(
  HMODULE hModule,
  ULONG ulReason,
  __in_opt PVOID pReserved)
{
  BOOL fRetVal = TRUE;

  if (DLL_PROCESS_ATTACH == ulReason)
  {
      // Disable thread attach notifications
      fRetVal = DisableThreadLibraryCalls(hModule);
  }

  return fRetVal;
} // DllMain

STDAPI DllGetClassObject(
  __in REFCLSID rclsid,
  __in REFIID riid,
  __deref_out LPVOID FAR *ppv)
{
  HRESULT hr = S_OK;
  CClassFactory* pClassFactory = NULL;

  if (ppv)
  {
      *ppv = NULL;
  }
  else
  {
      hr = E_INVALIDARG;
  }

  if (S_OK == hr)
  {
      if (CLSID_fdAudio != rclsid)
      {
          hr = CLASS_E_CLASSNOTAVAILABLE;
      }
  }

  if (S_OK == hr)
  {
      pClassFactory = new(std::nothrow) CClassFactory;

      if (!pClassFactory)
      {
          hr =  E_OUTOFMEMORY;
      }
  }

  if (S_OK == hr)
  {
      hr = pClassFactory->QueryInterface(riid, ppv);
  }

  if (pClassFactory)
  {
      pClassFactory->Release();
  }

  return hr;
}  // DLLGetClassObject

HRESULT APIENTRY DllCanUnloadNow()
{
  return (g_cLockCount == 0) ? S_OK : S_FALSE;
}  // DllCanUnloadNow

} // extern "C"

Finally you will have to fix your ATL free com objects in their constructors to increment and decrement the DLL lock count. By calling these:

inline VOID IncModuleCount()
{
  InterlockedIncrement(&g_cLockCount);
}  // IncModuleCount

inline VOID DecModuleCount()
{
  InterlockedDecrement(&g_cLockCount);
}  // DecModuleCount

Hopefully this will get you one step closer to being ATL free.

ATL Removal: Part 2 – Safely Deleting ATL_NO_VTABLE

2009-10-22T14:25:00.000-07:00

As I was going through and purging ATL from my project, I ran into the macro ATL_NO_VTABLE. It appears that some ATL project generators slap that baby into your class definition.

class ATL_NO_VTABLE CYourClass :…

It turns out that it is an optimization “that prevents the vtable pointer from being initialized in the class's constructor and destructor. If the vtable pointer is prevented from being initialized in the class's constructor and destructor, the linker can eliminate the vtable and all of the functions to which it points. Expands to __declspec(novtable).”

http://msdn.microsoft.com/en-us/library/6h06t6s8%28VS.80%29.aspx

It sounds like a good optimization for ATL code since your object will be wrapped by CComObject anyway; however, with normal com code, having the vtable line up with IUnknown is important. In turning your ATL com object into to a regular com object, it is safe to just remove ATL_NO_VTABLE.

ATL Removal: Part 1 – Removing BEGIN_COM_MAP() and END_COM_MAP()

2009-10-21T16:50:00.000-07:00

There are some who feel that ATL is the best thing since sliced bread for COM programming. ATL can abstract a lot of the nitty gritty COM programming away from the COM programmer. My goal today is not to bash ATL. I think it can be a great time saver if used correctly for the right level of code. If you are programming UI, or an IE plugin, go right ahead and use ATL if you like. If you are writing lower level system APIs, please don’t. Also, ATL cannot be used for an excuse for not understanding COM programming or memory management. I have spent the last year fixing lots of bugs because someone didn’t understand what ATL was doing, things like: double frees, double releases, memory leaks, leaked references, and so on. In other words, ATL is not a sliver bullet for API usage ignorance. I don’t like using ATL because: writing straight COM code is not much harder, and ATL makes debugging more annoying. So, in the end for me, ATL does not buy me much, and is somewhat annoying. If you like ATL and know what it is really doing so you use it correctly, go ahead and use it. Independent of my no ATL preference, one API I am maintaining needs to be “low level” and therefore should not depend on ATL, so I am removing ATL from that API.

ATL wants to internally manage the ref counts to your ATL based com object. To do this, it wraps your com object using a template class called CComObject. CComObject basically adds two extra layers to your com object. For debugging, it does make things a little bit more annoying, but it is supposed to manage the lifetime of your object for you.

The first thing to do is to remove the COM_MAP macros needed to set up all of the CComObject hooks. When you have an ATL generated COM object, it creates for you something like this in your header:


BEGIN_COM_MAP(CYourInterface)
  COM_INTERFACE_ENTRY(IYourInterface)
END_COM_MAP()

Basically these macros help write your IUnknown implementation. Not really that big of a time saver as it is not that hard to implement IUnknown. The first step is to remove COM_MAP macro stuff from your header and swap it for the IUnknown definition. Here is one example that you see often:


STDMETHOD(QueryInterface)(IN REFIID riid, OUT void ** ppv);
STDMETHOD_(ULONG, AddRef)(void);
STDMETHOD_(ULONG, Release)(void);

Next, in your cpp file where you implement your com object, you will need to add the implementation of IUnknown. This too is almost boiler plate.


//////////////////////////////////////////////////////////////////////
// Implementation : IUnknown
//////////////////////////////////////////////////////////////////////

ULONG CYourInterface::AddRef()
{
  return InterlockedIncrement(&m_cRef);
}

ULONG CYourInterface::Release()
{
  LONG cRef = InterlockedDecrement(&m_cRef);

  if (0 == cRef)
  {
      delete this;
  }

  return cRef;
}

HRESULT CYourInterface::QueryInterface(REFIID riid, __deref_out_opt void **ppv)
{
  HRESULT hr = S_OK;

  if (ppv)
  {
      *ppv = NULL;
  }
  else
  {
      hr = E_INVALIDARG;
  }

  if (S_OK == hr)
  {
      if ((__uuidof(IUnknown) == riid)  (riid == __uuidof(IYourInterface)))
      {
          AddRef();
          *ppv = (IYourInterface *)this;
      }
      else
      {
          hr = E_NOINTERFACE;
      }
  }

  return hr;
}

Finally make sure that you add a:

LONG m_cRef;

Member in your class, and make sure you initialize it to 1 in your constructor.

m_cRef(1)

GPU Accelerated FDTD Using Cg

2009-10-13T16:53:00.000-07:00

Apparently there has been some interested in my old GPU accelerated FDTD code implemented in C and Cg. I also had a CUDA version that had a hard time meeting the performance of the Cg version. I will need to check when I get home to see if I can find the other versions. Here is one version that I found on one of my old sites. It looks like it is version 0.1 and about three years old. I know I have a better versions somewhere out there, but this will give you a "taste" until I can find the other versions. Still, there are some interesting aspects of this code. I used an interesting 3D volume packing scheme for 2D textures that you can read about in GPU Gems 2. Also a lot of the guts of this code initially came from Dom's basic math tutorial for Cg.


cg_fdtd: clean
 gcc test.c -o cg_fdtd -lglut -lGLEW -lCgGL -lpthread

clean:
 rm -f cg_fdtd

test: cg_fdtd
 ./cg_fdtd 256 10


/*
 * test.c
 *
 * author      : Sam Adams
 * date        : 20061027
 * discription : This is my implementation of a basic FDTD using GPU with Cg
 */

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#define GLUT_WINDOW_NAME "fdtd window"

typedef enum bool{false, true} bool;
//clock_gettime
// struct for variable parts of GL calls (texture format, float format etc)
struct struct_textureParameters {
 char* name;
 GLenum texTarget;
 GLenum texInternalFormat;
 GLenum texFormat;
 char* shader_source;
};

// struct actually being used (set from command line)
struct struct_textureParameters textureParameters;

int verbose = 1;
int numIterations = 100; //timesteps for fdtd
int texSize = 256;

GLuint glutWindowHandle;
GLuint fb;

enum components{X, Y, Z};

GLuint *exTexID; // these are z slices for fdtd
GLuint *eyTexID; // the +1 is for the output of the calculations
GLuint *ezTexID;
GLuint *hxTexID;
GLuint *hyTexID;
GLuint *hzTexID;
GLuint psTexID, psEmptyTexID;

// Cg vars
CGcontext cgContext;
CGprofile fragmentProfile;
CGprogram fragmentProgram;
//fdtd
////for e fields
CGprogram ex_fp, ey_fp, ez_fp; 
CGparameter ex_ps, ex_e, ex_h1, ex_h2, ex_h3, ex_h4, ex_esctc, ex_eincc, ex_ei, ex_edevcn, ex_dei, ex_ecrl1, ex_ecrl2;
CGparameter ey_e, ey_h1, ey_h2, ey_h3, ey_h4, ey_esctc, ey_eincc, ey_ei, ey_edevcn, ey_dei, ey_ecrl1, ey_ecrl2;
CGparameter ez_e, ez_h1, ez_h2, ez_h3, ez_h4, ez_esctc, ez_eincc, ez_ei, ez_edevcn, ez_dei, ez_ecrl1, ez_ecrl2;
////for h fields
CGprogram hx_fp, hy_fp, hz_fp;
CGparameter hx_h, hx_e1, hx_e2, hx_e3, hx_e4, hx_dt, hx_delta1, hx_delta2;
CGparameter hy_h, hy_e1, hy_e2, hy_e3, hy_e4, hy_dt, hy_delta1, hy_delta2;
CGparameter hz_h, hz_e1, hz_e2, hz_e3, hz_e4, hz_dt, hz_delta1, hz_delta2;

float* tmpData;
//fdtd Cg update sources
char *hx_hUpdate_source = \
"float hUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT textureH, "\
"uniform samplerRECT textureE1, "\
"uniform samplerRECT textureE2, "\
"uniform samplerRECT textureE3, "\
"uniform samplerRECT textureE4, "\
"uniform float dt, "\
"uniform float delta1, "\
"uniform float delta2) : COLOR{ "\
"float h = texRECT(textureH, coords); "\
"float e1 = texRECT(textureE1, coords); "\
"float e2 = texRECT(textureE2, coords); "\
"float e3 = texRECT(textureE3, coords+(0.0,1.0)); "\
"float e4 = texRECT(textureE4, coords); "\
"return h + ((dt / 0.0000012566306) *((e1 - e2) / delta1 - (e3 - e4) / delta2)); }";

char *hy_hUpdate_source = \
"float hUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT textureH, "\
"uniform samplerRECT textureE1, "\
"uniform samplerRECT textureE2, "\
"uniform samplerRECT textureE3, "\
"uniform samplerRECT textureE4, "\
"uniform float dt, "\
"uniform float delta1, "\
"uniform float delta2) : COLOR{ "\
"float h = texRECT(textureH, coords); "\
"float e1 = texRECT(textureE1, coords+(1.0,0.0)); "\
"float e2 = texRECT(textureE2, coords); "\
"float e3 = texRECT(textureE3, coords); "\
"float e4 = texRECT(textureE4, coords); "\
"return h + ((dt / 0.0000012566306) *((e1 - e2) / delta1 - (e3 - e4) / delta2)); }";

char *hz_hUpdate_source = \
"float hUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT textureH, "\
"uniform samplerRECT textureE1, "\
"uniform samplerRECT textureE2, "\
"uniform samplerRECT textureE3, "\
"uniform samplerRECT textureE4, "\
"uniform float dt, "\
"uniform float delta1, "\
"uniform float delta2) : COLOR{ "\
"float h = texRECT(textureH, coords); "\
"float e1 = texRECT(textureE1, coords+(0.0,1.0)); "\
"float e2 = texRECT(textureE2, coords); "\
"float e3 = texRECT(textureE3, coords+(1.0,0.0)); "\
"float e4 = texRECT(textureE4, coords); "\
"return h + ((dt / 0.0000012566306) *((e1 - e2) / delta1 - (e3 - e4) / delta2)); }";

char *ex_eUpdate_source = \
"float eUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT texturePS, "\
"uniform samplerRECT textureE, "\
"uniform samplerRECT textureH1, "\
"uniform samplerRECT textureH2, "\
"uniform samplerRECT textureH3, "\
"uniform samplerRECT textureH4, "\
"uniform float esctc, "\
"uniform float eincc, "\
"uniform float ei, "\
"uniform float edevcn, "\
"uniform float dei, "\
"uniform float ecrl1, "\
"uniform float ecrl2 ) : COLOR{ "\
"float ps = texRECT(texturePS, coords); "\
"float e = texRECT(textureE, coords); "\
"float h1 = texRECT(textureH1, coords); "\
"float h2 = texRECT(textureH2, coords+(0.0,1.0)); "\
"float h3 = texRECT(textureH3, coords); "\
"float h4 = texRECT(textureH4, coords); "\
"float tmp = ps;"\
"if(ps != 0.0) tmp = ps;"\
/*"else tmp = e * esctc - eincc * ei - edevcn * dei + (h1 - h2) * ecrl1 - (h3 - h4) * ecrl2;"\*/
"else tmp = e * esctc - eincc * ei - edevcn * dei + ((h1 - h2) * ecrl1) - ((h3 - h4) * ecrl2);"\
"return tmp;}";

//"return e * esctc - eincc * ei - edevcn * dei + (h1 - h2) * ecrl1 - (h3 - h4) * ecrl2;}";
//"return e * esctc - eincc * ei - edevcn * dei + (h1 -h2) * ecrl1;}";

char *ey_eUpdate_source = \
"float eUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT textureE, "\
"uniform samplerRECT textureH1, "\
"uniform samplerRECT textureH2, "\
"uniform samplerRECT textureH3, "\
"uniform samplerRECT textureH4, "\
"uniform float esctc, "\
"uniform float eincc, "\
"uniform float ei, "\
"uniform float edevcn, "\
"uniform float dei, "\
"uniform float ecrl1, "\
"uniform float ecrl2 ) : COLOR{ "\
"float e = texRECT(textureE, coords); "\
"float h1 = texRECT(textureH1, coords); "\
"float h2 = texRECT(textureH2, coords); "\
"float h3 = texRECT(textureH3, coords); "\
"float h4 = texRECT(textureH4, coords+(1.0,0.0)); "\
"return e * esctc - eincc * ei - edevcn * dei + (h1 - h2) * ecrl1 - (h3 - h4) * ecrl2; }";

char *ez_eUpdate_source = \
"float eUpdate ("\
"in float2 coords : TEXCOORD0, "\
"uniform samplerRECT textureE, "\
"uniform samplerRECT textureH1, "\
"uniform samplerRECT textureH2, "\
"uniform samplerRECT textureH3, "\
"uniform samplerRECT textureH4, "\
"uniform float esctc, "\
"uniform float eincc, "\
"uniform float ei, "\
"uniform float edevcn, "\
"uniform float dei, "\
"uniform float ecrl1, "\
"uniform float ecrl2 ) : COLOR{ "\
"float e = texRECT(textureE, coords); "\
"float h1 = texRECT(textureH1, coords); "\
"float h2 = texRECT(textureH2, coords+(1.0,0.0)); "\
"float h3 = texRECT(textureH3, coords); "\
"float h4 = texRECT(textureH4, coords+(0.0,1.0)); "\
"return e * esctc - eincc * ei - edevcn * dei + (h1 - h2) * ecrl1 - (h3 - h4) * ecrl2; }";

time_t start, end;

/* prototypes */
void initGLUT(int argc, char **argv);
void initGLEW();
void initFBO();
void createTextures();
void initCG();
void performComputation();
void checkGLErrors (const char *label);
double cpuBench();
void printVector(float *v, int len);
void transferFromTexture(float* data, GLenum fb);
void initMemory();
//void nextSlice(int i);
void transferToTexture(float* data, GLuint texID);

int main(int argc, char **argv){
 int i;
 time_t start, stop;
 float simTime;
 double mflops;
 FILE *f;

 f = fopen("results", "a");

 texSize = atoi(argv[1]);
 numIterations = atoi(argv[2]);

// int start, stop;
// struct timespec res;
// clock_getres(CLOCK_REALTIME, &res);
 textureParameters.name                               = "TEXRECT - float_NV - R - 32";
 textureParameters.texTarget                          = GL_TEXTURE_RECTANGLE_ARB;
 textureParameters.texInternalFormat               = GL_FLOAT_R32_NV;
 textureParameters.texFormat                          = GL_LUMINANCE;
// start =  clock_gettime(CLOCK_REALTIME, &res);
 start = clock();
 initMemory();
 if(verbose) fprintf(stderr,"initalizing GLUT\n");
 initGLUT(argc, argv);
 if(verbose) fprintf(stderr,"initalizing GLEW\n");
 initGLEW();
 if(verbose) fprintf(stderr,"initalizing FBOs\n");
 initFBO();
 if(verbose) fprintf(stderr,"creating textures\n");
 createTextures();
 if(verbose) fprintf(stderr,"initalizing Cg\n");
 initCG();
 if(verbose) fprintf(stderr,"performing calculations\n");
 performComputation();
 stop = clock();
 simTime = (float)(stop-start)/(float)CLOCKS_PER_SEC;
 mflops = (double)(63*texSize*texSize*texSize*numIterations)/1000000.0;
 printf("time was %f\n", simTime);
 fprintf(f,"%i\t%i\t%e\t%e\n",texSize,numIterations,simTime,mflops);
 return 0;
}

void initMemory(){
 tmpData =  (float*)calloc(sizeof(float),texSize*texSize); // to initalize textures to 0.0
 exTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
 eyTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
 ezTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
 hxTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
 hyTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
 hzTexID = (GLuint*)malloc(sizeof(GLuint) * texSize);
/*
 exLocation = (int*)malloc(sizeof(int)*texSize);
 eyLocation = (int*)malloc(sizeof(int)*texSize);
 ezLocation = (int*)malloc(sizeof(int)*texSize);
 hxLocation = (int*)malloc(sizeof(int)*texSize);
 hyLocation = (int*)malloc(sizeof(int)*texSize);
 hzLocation = (int*)malloc(sizeof(int)*texSize);

 exLocation_out = eyLocation_out = ezLocation_out = hxLocation_out = hyLocation_out = hzLocation_out = texSize;*/
}

void printVector(float *v, int len){
 int i;

 for(i = 0; i < len; i++) printf("%i)\t%f\n",i,v[i]);
}

/*
 * Checks framebuffer status.
 * Copied directly out of the spec, modified to deliver a return value.
 */
int checkFramebufferStatus() {
 GLenum status;
 status = (GLenum) glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
 switch(status) {
  case GL_FRAMEBUFFER_COMPLETE_EXT:
   printf("Framebuffer complete\n");
   return 1;
  case GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT:
   printf("Framebuffer incomplete, incomplete attachment\n");
   return 0;
  case GL_FRAMEBUFFER_UNSUPPORTED_EXT:
   printf("Unsupported framebuffer format\n");
   return 0;
  case GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT:
   printf("Framebuffer incomplete, missing attachment\n");
   return 0;
  case GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT:
   printf("Framebuffer incomplete, attached images must have same dimensions\n");
   return 0;
  case GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT:
   printf("Framebuffer incomplete, attached images must have same format\n");
   return 0;
  case GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT:
   printf("Framebuffer incomplete, missing draw buffer\n");
   return 0;
  case GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT:
   printf("Framebuffer incomplete, missing read buffer\n");
   return 0;
  default:
   printf("Unknown framebuffer status %i\n", (int)status);
 }
 return 0;
}

void checkGLErrors (const char *label) {
 GLenum errCode;
 const GLubyte *errStr;

 if ((errCode = glGetError()) != GL_NO_ERROR) {
  errStr = gluErrorString(errCode);
  printf("OpenGL ERROR: ");
  printf((char*)errStr);
  printf("(Label: ");
  printf(label);
  printf(")\n.");
 }
}

/*
 * Performs the actual calculation.
 */
void performComputation(){
 int i, j;
 int tmp;
 double total;
 double mflops;

 start = clock();
 glEnable(textureParameters.texTarget);
 // attach textures to FBO
 glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, textureParameters.texTarget, exTexID[texSize], 0);
 glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT1_EXT, textureParameters.texTarget, eyTexID[texSize], 0);
 glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT2_EXT, textureParameters.texTarget, ezTexID[texSize], 0);
 // check if that worked
 if(!checkFramebufferStatus()) {
  printf("glFramebufferTexture2DEXT():\t [FAIL]\n");
  exit (-1);
 }// else if (mode == 0) {
 // printf("glFramebufferTexture2DEXT():\t [PASS]\n");
 // }
 // enable fragment profile
 cgGLEnableProfile(fragmentProfile);
 // bind fdtd program
 cgSetParameter1f(ex_esctc, 1.0);
 cgSetParameter1f(ex_eincc, 1.0);
 cgSetParameter1f(ex_ei, 1.0);
 cgSetParameter1f(ex_edevcn, 0.0);
 cgSetParameter1f(ex_dei, 0.0);
 cgSetParameter1f(ex_ecrl1, 217.51);
 cgSetParameter1f(ex_ecrl2, 217.51);

 cgGLBindProgram(ey_fp);
 cgSetParameter1f(ey_esctc, 1.0);
 cgSetParameter1f(ey_eincc, 0.0);
 cgSetParameter1f(ey_ei, 0.0);
 cgSetParameter1f(ey_edevcn, 0.0);
 cgSetParameter1f(ey_dei, 0.0);
 cgSetParameter1f(ey_ecrl1, 217.51);
 cgSetParameter1f(ey_ecrl2, 217.51);

 cgGLBindProgram(ez_fp);
 cgSetParameter1f(ez_esctc, 1.0);
 cgSetParameter1f(ez_eincc, 0.0);
 cgSetParameter1f(ez_ei, 0.0);
 cgSetParameter1f(ez_edevcn, 0.0);
 cgSetParameter1f(ez_dei, 0.0);
 cgSetParameter1f(ez_ecrl1, 217.51);
 cgSetParameter1f(ez_ecrl2, 217.51);

 cgGLBindProgram(hx_fp);
 cgSetParameter1f(hx_dt, 1.0);
 cgSetParameter1f(hx_delta1, 1.0);
 cgSetParameter1f(hx_delta2, 1.0);

 cgGLBindProgram(hy_fp);
 cgSetParameter1f(hy_dt, 1.0);
 cgSetParameter1f(hy_delta1, 1.0);
 cgSetParameter1f(hy_delta2, 1.0);

 cgGLBindProgram(hz_fp);
 cgSetParameter1f(hz_dt, 1.0);
 cgSetParameter1f(hz_delta1, 1.0);
 cgSetParameter1f(hz_delta2, 1.0);

 // sutff that changes should be in the loop...

 for(j = 0; j < numIterations; j++){
  //printf("iteration %i\n",j);
  for(i = 0; i < texSize; i++){
   //update x e field
   cgGLBindProgram(ex_fp);
//   if(i == texSize/2){
//    cgGLSetTextureParameter(ex_ps, psTexID);
//    cgGLEnableTextureParameter(ex_ps);
//   }
//   else{
//    cgGLSetTextureParameter(ex_ps, psEmptyTexID);
//    cgGLEnableTextureParameter(ex_ps);
//   }
   cgGLSetTextureParameter(ex_e, exTexID[i]);
   cgGLEnableTextureParameter(ex_e);
   cgGLSetTextureParameter(ex_h1, hzTexID[i]);
   cgGLEnableTextureParameter(ex_h1);
   cgGLSetTextureParameter(ex_h2, hzTexID[i]);
   cgGLEnableTextureParameter(ex_h2);
   cgGLSetTextureParameter(ex_h3, hyTexID[i]);
   cgGLEnableTextureParameter(ex_h3);
//   fprintf(stdout,"we are doing --i- %i --i-1- %i  %i\n",i, i-1,((i-1 <= 0) ? 0 : i-1)); 
   //if(i) cgGLSetTextureParameter(ex_h4, hyTexID[i-1]);
   if(i > 0) cgGLSetTextureParameter(ex_h4, hyTexID[i]);
   else cgGLSetTextureParameter(ex_h4, hyTexID[0]);
   cgGLEnableTextureParameter(ex_h4);
   glDrawBuffer(GL_COLOR_ATTACHMENT0_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
//    transferFromTexture(tmpData, hyTexID[i]);
   /*if(i == texSize/2){*/// fprintf(stdout,"h%i) hy ps %e\n",i,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
   //}
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT0_EXT);
   transferToTexture(tmpData, exTexID[i]);
//   if(i == texSize/2){ fprintf(stdout,"e%i) ex ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
   //update y e field
   cgGLBindProgram(ey_fp);
   cgGLSetTextureParameter(ey_e, eyTexID[i]);
   cgGLEnableTextureParameter(ey_e);
   cgGLSetTextureParameter(ey_h1, hxTexID[i]);
   cgGLEnableTextureParameter(ey_h1);
   cgGLSetTextureParameter(ey_h2, hxTexID[((i-1 < 0) ? 0 : i-1)]);
   cgGLEnableTextureParameter(ey_h2);
   cgGLSetTextureParameter(ey_h3, hzTexID[i]);
   cgGLEnableTextureParameter(ey_h3);
   cgGLSetTextureParameter(ey_h4, hyTexID[i]);
   cgGLEnableTextureParameter(ey_h4);
   glDrawBuffer(GL_COLOR_ATTACHMENT1_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT1_EXT);
   transferToTexture(tmpData, eyTexID[i]);

//   if(i == texSize/2){ fprintf(stdout,"e%i) ey ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
   //update z e field
   cgGLBindProgram(ez_fp);
   cgGLSetTextureParameter(ez_e, ezTexID[i]);
   cgGLEnableTextureParameter(ez_e);
   cgGLSetTextureParameter(ez_h1, hyTexID[i]);
   cgGLEnableTextureParameter(ez_h1);
   cgGLSetTextureParameter(ez_h2, hyTexID[i]);
   cgGLEnableTextureParameter(ez_h2);
   cgGLSetTextureParameter(ez_h3, hxTexID[i]);
   cgGLEnableTextureParameter(ez_h3);
   cgGLSetTextureParameter(ez_h4, hxTexID[i]);
   cgGLEnableTextureParameter(ez_h4);
   glDrawBuffer(GL_COLOR_ATTACHMENT2_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT2_EXT);
   transferToTexture(tmpData, ezTexID[i]);

//   if(i == texSize/2){ fprintf(stdout,"e%i) ez ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
   //update x h field
   cgGLBindProgram(hx_fp);
   cgGLSetTextureParameter(hx_h, hxTexID[i]);
   cgGLEnableTextureParameter(hx_h);
   cgGLSetTextureParameter(hx_e1, eyTexID[i+1]);
   cgGLEnableTextureParameter(hx_e1);
   cgGLSetTextureParameter(hx_e2, eyTexID[i]);
   cgGLEnableTextureParameter(hx_e2);
   cgGLSetTextureParameter(hx_e3, ezTexID[i]);
   cgGLEnableTextureParameter(hx_e3);
   cgGLSetTextureParameter(hx_e4, ezTexID[i]);
   cgGLEnableTextureParameter(hx_e4);
   glDrawBuffer(GL_COLOR_ATTACHMENT0_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT0_EXT);
   transferToTexture(tmpData, hxTexID[i]);
//   if(i == texSize/2){ fprintf(stdout,"h%i) hx ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
   
   //update y h field
   cgGLBindProgram(hy_fp);
   cgGLSetTextureParameter(hy_h, hyTexID[i]);
   cgGLEnableTextureParameter(hy_h);
   cgGLSetTextureParameter(hy_e1, ezTexID[i]);
   cgGLEnableTextureParameter(hy_e1);
   cgGLSetTextureParameter(hy_e2, ezTexID[i]);
   cgGLEnableTextureParameter(hy_e2);
   cgGLSetTextureParameter(hy_e3, exTexID[i+1]);
   cgGLEnableTextureParameter(hy_e3);
   cgGLSetTextureParameter(hy_e4, exTexID[i]);
   cgGLEnableTextureParameter(hy_e4);
   glDrawBuffer(GL_COLOR_ATTACHMENT1_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT1_EXT);
   transferToTexture(tmpData, hyTexID[i]);

//   if(i == texSize/2){ fprintf(stdout,"h%i) hy ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
   //update z h field
   cgGLBindProgram(hz_fp);
   cgGLSetTextureParameter(hz_h, hzTexID[i]);
   cgGLEnableTextureParameter(hz_h);
   cgGLSetTextureParameter(hz_e1, exTexID[i]);
   cgGLEnableTextureParameter(hz_e1);
   cgGLSetTextureParameter(hz_e2, exTexID[i]);
   cgGLEnableTextureParameter(hz_e2);
   cgGLSetTextureParameter(hz_e3, eyTexID[i]);
   cgGLEnableTextureParameter(hz_e3);
   cgGLSetTextureParameter(hz_e4, eyTexID[i]);
   cgGLEnableTextureParameter(hz_e4);
   glDrawBuffer(GL_COLOR_ATTACHMENT2_EXT);
   glPolygonMode(GL_FRONT,GL_FILL);
   //render/compute 
   glBegin(GL_QUADS);
   glTexCoord2f(0.0, 0.0);
   glVertex2f(0.0, 0.0);
   glTexCoord2f(texSize, 0.0);
   glVertex2f(texSize, 0.0);
   glTexCoord2f(texSize, texSize);
   glVertex2f(texSize, texSize);
   glTexCoord2f(0.0, texSize);
   glVertex2f(0.0, texSize);
   glEnd();
   //get result
   transferFromTexture(tmpData, GL_COLOR_ATTACHMENT2_EXT);
   transferToTexture(tmpData, hzTexID[i]);
//   if(i == texSize/2){ fprintf(stdout,"h%i) hz ps %e\n",j,tmpData[(texSize*texSize)/2 + 1]);
//    printVector(tmpData, texSize*texSize);
//   }
  }
 }
 // done, stop timer, calc MFLOP/s if neccessary
// glFinish();
// end = clock();
// total = (double)(end-start)/(double)CLOCKS_PER_SEC;
// mflops = (double)((3*12 + 3*9)*texSize*texSize*texSize*numIterations) / (total * 1000000.0);
// printf("GPU MFLOP/s for N=%d:\t\t%f\n",texSize, mflops);
 // done, just do some checks if everything went smoothly.
 checkFramebufferStatus();
 checkGLErrors("render()");
 glDisable(textureParameters.texTarget);
// transferFromTexture(tmpData);
// printVector(tmpData, texSize*texSize);
 //printVector(dataX, texSize*texSize);

 // do cpu comarison
// printf("GPU speedup %f\n \n", cpuBench()/total);
}

void transferFromTexture(float* data, GLenum fb){
 // version (a): texture is attached
 // recommended on both NVIDIA and ATI
 glReadBuffer(fb);
 glReadPixels(0, 0, texSize, texSize,textureParameters.texFormat,GL_FLOAT,data);
}

void cgErrorCallback(){
 CGerror lastError = cgGetError();
 if(lastError) {
  printf(cgGetErrorString(lastError));
  printf(cgGetLastListing(cgContext));
  exit(lastError);
 }
}

/*
 * Sets up the Cg runtime and creates shader.
 */
void initCG(void) {
 // set up Cg
 cgSetErrorCallback(cgErrorCallback);
 cgContext = cgCreateContext();
 fragmentProfile = cgGLGetLatestProfile(CG_GL_FRAGMENT);
 cgGLSetOptimalOptions(fragmentProfile);
 // create fragment program
 ex_fp = cgCreateProgram(cgContext, CG_SOURCE, ex_eUpdate_source, fragmentProfile, "eUpdate", NULL);
 ey_fp = cgCreateProgram(cgContext, CG_SOURCE, ey_eUpdate_source, fragmentProfile, "eUpdate", NULL);
 ez_fp = cgCreateProgram(cgContext, CG_SOURCE, ez_eUpdate_source, fragmentProfile, "eUpdate", NULL);

 hx_fp = cgCreateProgram(cgContext, CG_SOURCE, hx_hUpdate_source, fragmentProfile, "hUpdate", NULL);
 hy_fp = cgCreateProgram(cgContext, CG_SOURCE, hy_hUpdate_source, fragmentProfile, "hUpdate", NULL);
 hz_fp = cgCreateProgram(cgContext, CG_SOURCE, hz_hUpdate_source, fragmentProfile, "hUpdate", NULL);
// // load programs
 cgGLLoadProgram(ex_fp);
 cgGLLoadProgram(ey_fp);
 cgGLLoadProgram(ez_fp);
 cgGLLoadProgram(hx_fp);
 cgGLLoadProgram(hy_fp);
 cgGLLoadProgram(hz_fp);
 //and get parameter handles by name
 //ex
 ex_ps = cgGetNamedParameter(ex_fp, "texturePS");
 ex_e = cgGetNamedParameter(ex_fp, "textureE");
 ex_h1 = cgGetNamedParameter(ex_fp, "textureH1");
 ex_h2 = cgGetNamedParameter(ex_fp, "textureH2");
 ex_h3 = cgGetNamedParameter(ex_fp, "textureH3");
 ex_h4 = cgGetNamedParameter(ex_fp, "textureH4");
 ex_esctc = cgGetNamedParameter(ex_fp, "esctc");
 ex_eincc = cgGetNamedParameter(ex_fp, "eincc");
 ex_ei = cgGetNamedParameter(ex_fp, "ei");
 ex_edevcn = cgGetNamedParameter(ex_fp, "edevcn");
 ex_dei = cgGetNamedParameter(ex_fp, "dei");
 ex_ecrl1 = cgGetNamedParameter(ex_fp, "ecrl1");
 ex_ecrl2 = cgGetNamedParameter(ex_fp, "ecrl2");
 //ey
 ey_e = cgGetNamedParameter(ey_fp, "textureE");
 ey_h1 = cgGetNamedParameter(ey_fp, "textureH1");
 ey_h2 = cgGetNamedParameter(ey_fp, "textureH2");
 ey_h3 = cgGetNamedParameter(ey_fp, "textureH3");
 ey_h4 = cgGetNamedParameter(ey_fp, "textureH4");
 ey_esctc = cgGetNamedParameter(ey_fp, "esctc");
 ey_eincc = cgGetNamedParameter(ey_fp, "eincc");
 ey_ei = cgGetNamedParameter(ey_fp, "ei");
 ey_edevcn = cgGetNamedParameter(ey_fp, "edevcn");
 ey_dei = cgGetNamedParameter(ey_fp, "dei");
 ey_ecrl1 = cgGetNamedParameter(ey_fp, "ecrl1");
 ey_ecrl2 = cgGetNamedParameter(ey_fp, "ecrl2");
 //ez
 ez_e = cgGetNamedParameter(ez_fp, "textureE");
 ez_h1 = cgGetNamedParameter(ez_fp, "textureH1");
 ez_h2 = cgGetNamedParameter(ez_fp, "textureH2");
 ez_h3 = cgGetNamedParameter(ez_fp, "textureH3");
 ez_h4 = cgGetNamedParameter(ez_fp, "textureH4");
 ez_esctc = cgGetNamedParameter(ez_fp, "esctc");
 ez_eincc = cgGetNamedParameter(ez_fp, "eincc");
 ez_ei = cgGetNamedParameter(ez_fp, "ei");
 ez_edevcn = cgGetNamedParameter(ez_fp, "edevcn");
 ez_dei = cgGetNamedParameter(ez_fp, "dei");
 ez_ecrl1 = cgGetNamedParameter(ez_fp, "ecrl1");
 ez_ecrl2 = cgGetNamedParameter(ez_fp, "ecrl2");
 //hx
 hx_h = cgGetNamedParameter(hx_fp, "textureH");
 hx_e1 = cgGetNamedParameter(hx_fp, "textureE1");
 hx_e2 = cgGetNamedParameter(hx_fp, "textureE2");
 hx_e3 = cgGetNamedParameter(hx_fp, "textureE3");
 hx_e4 = cgGetNamedParameter(hx_fp, "textureE4");
 hx_dt = cgGetNamedParameter(hx_fp, "dt");
 hx_delta1 = cgGetNamedParameter(hx_fp, "delta1");
 hx_delta2 = cgGetNamedParameter(hx_fp, "delta2");
 //hy
 hy_h = cgGetNamedParameter(hy_fp, "textureH");
 hy_e1 = cgGetNamedParameter(hy_fp, "textureE1");
 hy_e2 = cgGetNamedParameter(hy_fp, "textureE2");
 hy_e3 = cgGetNamedParameter(hy_fp, "textureE3");
 hy_e4 = cgGetNamedParameter(hy_fp, "textureE4");
 hy_dt = cgGetNamedParameter(hy_fp, "dt");
 hy_delta1 = cgGetNamedParameter(hy_fp, "delta1");
 hy_delta2 = cgGetNamedParameter(hy_fp, "delta2");
 //hz
 hz_h = cgGetNamedParameter(hz_fp, "textureH");
 hz_e1 = cgGetNamedParameter(hz_fp, "textureE1");
 hz_e2 = cgGetNamedParameter(hz_fp, "textureE2");
 hz_e3 = cgGetNamedParameter(hz_fp, "textureE3");
 hz_e4 = cgGetNamedParameter(hz_fp, "textureE4");
 hz_dt = cgGetNamedParameter(hz_fp, "dt");
 hz_delta1 = cgGetNamedParameter(hz_fp, "delta1");
 hz_delta2 = cgGetNamedParameter(hz_fp, "delta2");
}

/*
 * Transfers data to texture. 
 * Check web page for detailed explanation on the difference between ATI and NVIDIA.
 */
void transferToTexture (float* data, GLuint texID) {
 // version (a): HW-accelerated on NVIDIA 
 glBindTexture(textureParameters.texTarget, texID);
 glTexSubImage2D(textureParameters.texTarget,0,0,0,texSize,texSize,textureParameters.texFormat,GL_FLOAT,data);
}

/*
 * Sets up a floating point texture with NEAREST filtering.
 * (mipmaps etc. are unsupported for floating point textures)
 */
void setupTexture (const GLuint texID) {
 // make active and bin
 int err;
 glBindTexture(textureParameters.texTarget,texID);
 // turn off filtering and wrap modes
 glTexParameteri(textureParameters.texTarget, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
 glTexParameteri(textureParameters.texTarget, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
 glTexParameteri(textureParameters.texTarget, GL_TEXTURE_WRAP_S, GL_CLAMP);
 glTexParameteri(textureParameters.texTarget, GL_TEXTURE_WRAP_T, GL_CLAMP);
 // define texture with floating point format
 glTexImage2D(textureParameters.texTarget,0,textureParameters.texInternalFormat,texSize,texSize,0,textureParameters.texFormat,GL_FLOAT,0);
 // define texture with floating point format
 // check if that worked
 if(err = glGetError() != GL_NO_ERROR){
  printf("glTexImage2D():\t\t\t [FAIL]\n");
  exit(err);
 }
// else if(mode == 0){
  printf("glTexImage2D():\t\t\t [PASS]\n");
// }
 printf("Created a %i by %i floating point texture.\n",texSize,texSize);
}

void createTextures(){
 int i;
 // create textures 
 glGenTextures(texSize+1, exTexID);
 glGenTextures(texSize+1, eyTexID);
 glGenTextures(texSize+1, ezTexID);
 glGenTextures(texSize+1, hxTexID);
 glGenTextures(texSize+1, hyTexID);
 glGenTextures(texSize+1, hzTexID);
 glGenTextures(1, &psTexID);
 glGenTextures(1, &psEmptyTexID);

 tmpData[(texSize*texSize)/2] = 5.0;
 setupTexture(psTexID);
 transferToTexture(tmpData, psTexID);
 tmpData[(texSize*texSize)/2] = 0.0;
 setupTexture(psEmptyTexID);
 transferToTexture(tmpData, psEmptyTexID);

 
 // set up textures
 for(i = 0; i <= texSize; i++){
  setupTexture(exTexID[i]);
  transferToTexture(tmpData, exTexID[i]);
  setupTexture(eyTexID[i]);
  transferToTexture(tmpData, eyTexID[i]);
  setupTexture(ezTexID[i]);
  transferToTexture(tmpData, ezTexID[i]);
  setupTexture(hxTexID[i]);
  transferToTexture(tmpData, hxTexID[i]);
  setupTexture(hyTexID[i]);
  transferToTexture(tmpData, hyTexID[i]);
  setupTexture(hzTexID[i]);
  transferToTexture(tmpData, hzTexID[i]);
 }
 // set texenv mode from modulate (the default) to replace)
 glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
 // check if something went completely wrong
 checkGLErrors ("createFBOandTextures()");
}

/*
 * Creates framebuffer object, binds it to reroute rendering operations 
 * from the traditional framebuffer to the offscreen buffer
 */
void initFBO(){
 // create FBO (off-screen framebuffer)
 glGenFramebuffersEXT(1, &fb);
 // bind offscreen framebuffer (that is, skip the window-specific render target)
 glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fb);
 // viewport for 1:1 pixel=texture mapping
 glMatrixMode(GL_PROJECTION);
 glLoadIdentity();
 gluOrtho2D(0.0, texSize, 0.0, texSize);
 glMatrixMode(GL_MODELVIEW);
 glLoadIdentity();
 glViewport(0, 0, texSize, texSize);
}

void initGLEW (void) {
 int err = glewInit();
 if (GLEW_OK != err) {
  fprintf(stderr,"error: %s\n",(char*)glewGetErrorString(err));
  exit(-1);
 }
}

void initGLUT(int argc, char **argv){
 glutInit(&argc, argv);
 glutWindowHandle = glutCreateWindow(GLUT_WINDOW_NAME);
}

How To Use the String Stream implementation of ISequentialStream

2009-10-13T14:04:00.000-07:00

Someone the other day asked how do they use the ISequentialStream string class I implemented.

Here is the original post on the string stream implementation.

This is how you would use it.


HRESULT hr = S_OK;
LPWSTR pszXml = NULL;
ISequentialStream *pStream = NULL;
IXmlReader *pReader = NULL;

hr = GetXmlString(&pszXml);

if (S_OK == hr)
{
   hr = CStringStream::Create(pszXml, &pStream);
}

if (S_OK == hr)
{
   hr = CreateXmlReader(__uuidof(IXmlReader), (void**)&pReader, NULL);
}

if (S_OK == hr)
{
   hr = pReader->SetInput(pStream);
}

//
// rest of your xml parsing here.
//

// cleanup
if (pReader)
{
   pReader->Release();
}

if (pStream)
{
   pStream->Release();
}

if (pszXml)
{
   free(pszXml);
}

Making Your Function Discovery (FD) Provider Run In Proc

2009-10-13T12:40:00.000-07:00

I was looking through my basic FD provider sample that I wrote a few months ago and realized that I can't directly publish it on the Internet. I don't want to provide a sample that doesn't do anything real, so I will need to rewrite a new provider sample that publishable and is realistic. In the mean time, if you want to push ahead and try writing your own provider, it is actually not that hard. First, you need to do the standard things for making a new DLL: DLLMain, DLLGetClassObject, implement IClassFactory, et cetera. Then, you need to implement IFunctionDiscoveryProvider. That is it. Well, to get it to work, you will also have to make the proper registrations (hint: use the in box providers registry keys as an example). The second problem will be that your provider will have to implement all of the PnPX stuff to be loaded out of proc because FD will only load known providers inproc by default. If you want to ship a provider, pnpx should be your end game especially if you are doing network type providers. While you are learning the basics, there is a trick to get your FD client to load your non pnpx provider inproc. Note, this is done on the client or provider consumer side. This is how you would do that. I will try to provide a complete sample later if time permits.

hr = pPnpQuery->AddQueryConstraint(
FD_QUERYCONSTRAINT_COMCLSCONTEXT,
FD_CONSTRAINTVALUE_COMCLSCONTEXT_INPROC_SERVER
);

Unit Testing

2009-10-13T12:28:00.000-07:00

Since we are between product cycles, they want us to be writing tests and not making changes on the product code until initial planning is done. Testing is like hygiene, not specifically fun, but necessary. Writing unit tests is not my favorite thing to do, but it is need if you want to have quality in your code. So, I went about figuring out how to setup good unit tests in our build environment and how to automate them. I won't go in to those details because they are unique to our environment, but I will provide some slides for a presentation that I gave to my team.

Unit Testing
What Is a Unit Test?
• Unit: smallest testable part of a program: functions, classes, methods
• Validates correct behavior of the unit
• Ideally independent of the other unit tests
• They should cover most code paths
• Generally a white box testing method that is close to the code implementation
• First line of testing
• Should be written in conjunction with the unit of code

What Isn’t Unit Testing?
• A catchall for every bug
• Replacement for other testing
• Functional testing: validates code functions to spec (higher level and more black box than unit tests)
• Integration testing: tests how the units are put together

Why Write Unit Tests?
• Finds bugs early in the development cycle
• Gives confidence that the units you are writing are behaving correctly
• Provides quick feedback if functionality has inadvertently been regressed
• Simplifies refactoring
• Gives confidence when you make late milestone changes
• Documents and defines correct unit behavior

Dev IC Workflow
• Spec & design
• Product code & unit tests
• Check in
• Automation

Unit Test Writing Work Flow
Test Driven Development
TDD Cycle
• Write a test
• Run all unit tests – the new test should fail
• Write some code – write enough code to pass the test
• Run all unit tests – the new test should now pass
• Refactor – clean up the code and tests as needed
• Repeat steps 1-5 until all code units are complete

TDD Benefits
• Promotes better design since you must think about using the API before writing them
• Heavy debugging is rarely needed
• Many of the bugs are found even before it is checked in
• Promotes good unit test coverage
• Discourages code creep
• Unit tests are easier to write before than after

Function Discovery: Callback Objects, Implementing IFunctionDiscoveryNotification

2009-09-15T13:56:00.000-07:00

If you missed my introductory post on Function Discovery (FD), you might want to go back there and give it a once over. It will give you a quick primer on what FD is about.

Function Discovery Intro

In my first FD post I provided a sample using the PnP FD provider to enumerate present devices. The FD PnP provider probably the most used provider and is easier to use than SetupDiGetGlassDevs especially if your program is already using COM. Unfortunately my first sample didn’t include a callback object which is required to get notifications from the provider. It gets worse than that; the PnP provider is actually the only provider (except the registry provider) that will provide synchronous results (IFunctionInstanceCollection)when you execute your query. In other words, every other inbox FD provider is asynchronous, and you won’t get any function instance (FI) results unless you provide a callback object and get them asynchronously.

Don’t worry; writing callback objects is easy, and I will show you how with an example. You start off creating a query just like we did in the first example, except we will have to change two parts. First you will need to create your callback object, and then pass it as a parameter to the CreateInstanceCollectionQuery method call. Finally when you execute the query, you will not get a function instance query back unless it is the PnP or registry provider, and the call will return E_PENDING. E_PENDING is not an error if you are using an asynchronous provider; it just means that the provider will give you function instances asynchronously to your call back object. If the provider is async and returns E_PENDING, it should also send FD_EVENTID_SEARCHCOMPLETE to the callback’s OnEvent method.

Here is a simple sample code for a callback object.


class CNotificationCallback : public IFunctionDiscoveryNotification
{
public:

    STDMETHODIMP_(ULONG) AddRef();

    STDMETHODIMP_(ULONG) Release();

    STDMETHODIMP QueryInterface(
        REFIID riid, 
        __deref_out_opt void **ppv);

    STDMETHODIMP OnUpdate(
        QueryUpdateAction enumQueryUpdateAction, 
        FDQUERYCONTEXT fdqcQueryContext, 
        __in IFunctionInstance *pIFunctionInstance);

    STDMETHODIMP OnError(
        HRESULT hr, 
        FDQUERYCONTEXT fdqcQueryContext, 
        PCWSTR pszProvider);

    STDMETHODIMP OnEvent(
        DWORD dwEventID, 
        FDQUERYCONTEXT fdqcQueryContext, 
        PCWSTR pszProvider);

    CNotificationCallback();

protected:
    LONG m_cRef;
};

CNotificationCallback::CNotificationCallback():
    m_cRef(1)
{
}

STDMETHODIMP_(ULONG) CNotificationCallback::AddRef()
{
    return InterlockedIncrement(&m_cRef);
}

STDMETHODIMP_(ULONG) CNotificationCallback::Release()
{
    LONG cRef = InterlockedDecrement(&m_cRef);
    if (0 == cRef)
    {
        delete this;
    }

    return cRef;
}

STDMETHODIMP CNotificationCallback::QueryInterface(
    REFIID riid, 
    __deref_out_opt void **ppv)
{
    HRESULT hr = S_OK;

    if (ppv)
    {
        *ppv = NULL;
    }
    else
    {
        hr = E_INVALIDARG;
    }

    if (S_OK == hr)
    {
        if (__uuidof(IUnknown) == riid )
        {
            AddRef();
            *ppv = (IUnknown*) this;
        }
        else if (__uuidof(IFunctionDiscoveryNotification) == riid)
        {
            AddRef();
            *ppv = (IFunctionDiscoveryNotification*) this;
        }
        else
        {
            hr = E_NOINTERFACE;
        }
    }

    return hr;
}

STDMETHODIMP CNotificationCallback::OnUpdate(
    QueryUpdateAction enumQueryUpdateAction, 
    FDQUERYCONTEXT fdqcQueryContext, 
    __in IFunctionInstance* pIFunctionInstance)
{
    HRESULT hr = S_OK;

    switch (enumQueryUpdateAction)
    {
    case QUA_ADD:
        wprintf(L"QUA_ADD\n");
        break;
    case QUA_REMOVE:
        wprintf(L"QUA_REMOVE\n");
        break;
    case QUA_CHANGE:
        wprintf(L"QUA_CHANGE\n");
        break;
    }

    return S_OK;
}

STDMETHODIMP CNotificationCallback::OnError(
    HRESULT hr, 
    FDQUERYCONTEXT fdqcQueryContext, 
    PCWSTR pszProvider)
{
    wprintf(L"****** ERROR: 0x%08x\n", hr);

    return S_OK;
}

STDMETHODIMP CNotificationCallback::OnEvent(
    DWORD dwEventID, 
    FDQUERYCONTEXT fdqcQueryContext, 
    PCWSTR pszProvider)
{
    wprintf(L"Event: %d\n", dwEventID);

    return S_OK;
}

This is the most basic example of how you might write a callback. Let’s pretend that we wanted to take an async provider like WSD and make it synchronized. One way you could do this is by passing in an empty function instance collection, and a handle that the callback object can signal once it receives FD_EVENTID_SEARCHCOMPLETE from the provider. In the main thread you could just wait on the handle. Often the callback interface is inherited in a bigger fancier class that does a lot more things than implement IFunctionDiscoveryNotification; the sky is the limit on how you want to structure your code here. Just make sure you exercise good tread safety. If you are sharing memory between the main program thread and your callback’s tread, be sure to use a SRW lock.

Now armed with callbacks, you can use two of FD’s main features: enumerating, and receiving notifications. With callbacks you will be able to take advantage of all of the providers on your computer.

Hopefully next time we can see how simple it is to write a FD provider and register it on your computer. Once we can write a simple provider, we can move on to writing full blown PnP-X providers. If you want to skip straight to PnP-X, there is a sample of one in the Windows SDK already, but hopefully I will be able to break it down into more digestible chunks. :)

If you are interested in using FD and want some extra help, email me and I can get you going on writing your provider or whatever you want to get accomplished.

Function Discovery Intro

2009-09-08T17:23:00.000-07:00

The other day I wrote a post about using SetupDi to enumerate PnP devices.

SetupDi Post

But SetupDi is not the only API to enumerate devices; Function Discovery can also enumerate PnP devices along with a host of other capabilities.

Function Discovery (FD) came to life as part of Windows Vista. FD’s main goal was to provide a unified API and interfaces for gathering functionality, properties, and notifications from various providers. PnP just happens to be one of the providers. Before FD, different API sets were required for discovering functionality of devices; for example you could use SetupDiGetClassDevs to find physically connected devices, but you had to use other APIs for network devices or printers. Using FD, you can use the same set of interfaces and methods for PnP and any number of devices exposed trough a provider. Vista shipped with in-box-providers for PnP, PnP-X (WSD & SSDP), Registry, NetBIOS, and the capability for third parties to create their own providers, and Windows 7 there are even more providers.

If you have the Windows SDK installed (I assume that you would if you are interested in writing this kind of code), you can do some header spelunking. Check out FunctionDiscoveryCategories.h to get an idea of what providers you can try to use. Also you can dig into the registry to see what other providers are registered on the system at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Function Discovery\Categories.

So if you need to enumerate devices and/or get notifications, FD can help assuming there is a provider for it.

To start out with, I wrote a sample FD code using the PnP provider to enumerate PnP devices just like I did in the SetupDiGetClassDevs example from before. I wrote this a while back now, so I hope there are no bug in this code.

I am having it print out a few properties from each function instance. A good place to go to find what kinds of properties are discoverable through FD is the header files included in the SDK: functiondiscoverykeys.h, and functiondiscoverykeys_devpkey.h. Not all PKEYs are populated by all providers; for example, PKEY_WSD_MetadataVersion will not be populated by the PnP provider, but will be populated by the WSD provider. Generally PKEYs populated by the PnP are prefixed with PKEY_Device_, and the properties you can get with SetupDi are available with the PnP FD provider.

For a more in depth reference to Function Discovery, refer to the official MSDN FD documentation.


/* displays pnp function instances */

#include <stdio.h>
#include <FunctionDiscovery.h>

HRESULT PrintFIs(IFunctionInstanceCollection* FIs);

int __cdecl wmain(
    __in int argc, 
    __in_ecount(argc) PWSTR
)
{
    HRESULT hr = S_OK;
    IFunctionDiscovery *pFD = NULL;
    IFunctionInstanceCollectionQuery *pPnpQuery = NULL;
    IFunctionInstanceCollection *pFICollection = NULL;

    hr = CoInitializeEx(NULL, COINIT_MULTITHREADED);

    // CoCreate FunctionDiscovery
    if (S_OK == hr)
    {
        hr = CoCreateInstance(
                __uuidof(FunctionDiscovery),
                NULL,
                CLSCTX_ALL,
                __uuidof(IFunctionDiscovery),
                (PVOID*)&pFD);
    }

    // Query the pnp provider
    if (S_OK == hr)
    {
        hr = pFD->CreateInstanceCollectionQuery(
                FCTN_CATEGORY_PNP, // pnp category (defined in functiondiscoverycategories.h)
                NULL,              // subcategory
                FALSE,             // include subcategories
                NULL,              // notification callback
                NULL,              // context
                &pPnpQuery);       // FI collection query
    }

    /*
    // * optional *
    // add property constraints
    // generally only works with core FD properties, and not provider specific properties
    if (S_OK == hr)
    {
        PROPVARIANT pv;

        PropVariantClear(&pv);

        pv.vt = VT_UINT;
        pv.uintVal = 0;

        hr = pPnpQuery->AddPropertyConstraint(PKEY_FD_Visibility, &pv, QC_EQUALS);

        PropVariantClear(&pv);
    }
    */

    /*
    // * optional *
    // add query constraints
    // refer to functiondiscoveryconstraints.h
    if (S_OK == hr)
    {
        hr = pPnpQuery->AddQueryConstraint(PNP_CONSTRAINTVALUE_NOTPRESENT, FD_CONSTRAINTVALUE_TRUE);
    }
    */

    if (S_OK == hr)
    {
        hr = pPnpQuery->Execute(&pFICollection);
    }

    if (S_OK == hr)
    {
        hr = PrintFIs(pFICollection);
    }

    // clean up
    if (pFD)
    {
        pFD->Release();
    }

    if (pPnpQuery)
    {
        pPnpQuery->Release();
    }

    if (pFICollection)
    {
        pFICollection->Release();
    }

    CoUninitialize();

    if (S_OK != hr)
    {
        wprintf(L"an error occured (hr == 0x%x)\n", hr);
        return 1;
    }

    return 0;
}

HRESULT PrintFIs(
    IFunctionInstanceCollection* FIs
)
{
    HRESULT hr = S_OK;
    DWORD cFIs = 0;
    IFunctionInstance *pFI = NULL;
    IFunctionInstanceCollection *pDeviceFunctionCollection = NULL;

    if (FIs)
    {
        hr = FIs->GetCount(&cFIs);

        wprintf(L"*******************************************************\n");
        wprintf(L"* %i Function Instances\n", cFIs);
        wprintf(L"*******************************************************\n\n");

        // go through each function instance
        for (DWORD i = 0; S_OK == hr && i < cFIs; i++)
        {
            hr = FIs->Item(i, &pFI);

            if (S_OK == hr)
            {
                IPropertyStore *pPropertyStore = NULL;

                pFI->OpenPropertyStore(STGM_READ, &pPropertyStore);

                if (pPropertyStore)
                {
                    PROPVARIANT pv;

                    PropVariantClear(&pv);

                    // PKEYs can be found in these headers in the SDK:
                    // functiondiscoverykeys.h functiondiscoverykeys_devpkey.h
                    // Providers do not populate all PKEYs.
                    hr = pPropertyStore->GetValue(PKEY_Device_FriendlyName, &pv);

                    if (S_OK == hr)
                    {
                        wprintf(L"Device Friendly Name : \"%s\"\n", (pv.vt == VT_LPWSTR) ? pv.pwszVal : L"");
                    }

                    PropVariantClear(&pv);


                    hr = pPropertyStore->GetValue(PKEY_Device_InstanceId, &pv);

                    if (S_OK == hr && VT_LPWSTR == pv.vt)
                    {
                        wprintf(L"\tDevice Instance ID : \"%s\"\n", pv.pwszVal);
                    }

                    PropVariantClear(&pv);

                    hr = pPropertyStore->GetValue(PKEY_Device_Class, &pv);

                    if (S_OK == hr && VT_LPWSTR == pv.vt)
                    {
                        wprintf(L"\tClass : %s",pv.pwszVal);
                    }

                    PropVariantClear(&pv);

                    hr = pPropertyStore->GetValue(PKEY_Device_ClassGuid, &pv);

                    if (S_OK == hr && VT_CLSID == pv.vt)
                    {
                        wprintf(L"\t(GUID : %x-%x-%x-%x%x-%x%x%x%x%x%x)\n",
                                pv.puuid->Data1, 
                                pv.puuid->Data2, 
                                pv.puuid->Data3, 
                                pv.puuid->Data4[0],
                                pv.puuid->Data4[1],
                                pv.puuid->Data4[2],
                                pv.puuid->Data4[3],
                                pv.puuid->Data4[4],
                                pv.puuid->Data4[5],
                                pv.puuid->Data4[6],
                                pv.puuid->Data4[7]
                                );
                    }

                    PropVariantClear(&pv);

                    pPropertyStore->Release();
                }
            }

            if (pFI)
            {
                pFI->Release();
                pFI = NULL;
            }

            if (pDeviceFunctionCollection)
            {
                pDeviceFunctionCollection->Release();
                pDeviceFunctionCollection = NULL;
            }
        }
    }

    return hr;
}

Creating an IStream or ISequentialStream From a String for XmlLite

2009-08-12T15:31:00.001-07:00

XmlLite needs an IStream or an ISequentialStream to parse from. You can get one by opening a file like I showed in the previous post, but in my real code I didn’t have a file, I had a string. No biggie, you can always implement your own if there isn’t one already. This CStringStream class implements ISequentialStream using a string as an input. The class factory method takes in a string, creates a buffer, and gives back an ISequentialStream. Awesome, just what you need if you want to use XmlLite on XML in a string. Here is the class implementation:


#pragma once

//
// this class creates an ISequentialStream from a string
//
class CStringStream : public ISequentialStream
{
public:
    // factory method
    __checkReturn static HRESULT Create(
            __in LPWSTR psBuffer,
            __deref_out ISequentialStream **ppStream) 
    {
        HRESULT hr = S_OK;
        void *pNewBuff = NULL;
        size_t buffSize = 0;

        if (!psBuffer)
        {
            return E_INVALIDARG;
        }

        *ppStream = NULL;

        buffSize = (wcslen(psBuffer)+1) * sizeof(wchar_t);
        pNewBuff = malloc(buffSize);

        if (!pNewBuff)
        {
            return E_OUTOFMEMORY;
        }

        hr = StringCbCopy((LPWSTR)pNewBuff, buffSize, psBuffer);

        if (S_OK == hr)
        {
            *ppStream = new CStringStream(
                    buffSize,
                    pNewBuff);
        }

        if (!*ppStream)
        {
            hr = E_FAIL;
        }

        return hr;
    };

    // ISequentialStream
    __checkReturn HRESULT STDMETHODCALLTYPE Read(
        __out_bcount_part(cb, *pcbRead) void *pv,
        /* [in] */ ULONG cb,
        __out_opt  ULONG *pcbRead)
    {
        HRESULT hr = S_OK;

        for (*pcbRead = 0; *pcbRead < cb; ++*pcbRead, ++m_buffSeekIndex)
        {
            // we are seeking past the end of the buffer
            if (m_buffSeekIndex == m_buffSize)
            {
                hr = S_FALSE;
                break;
            }
            
            ((BYTE*)pv)[*pcbRead] = ((BYTE*)m_pBuffer)[m_buffSeekIndex];
        }

        return hr;
    };

    HRESULT STDMETHODCALLTYPE Write(
        __in_bcount(cb)  const void *pv,
        /* [in] */ ULONG cb,
        __out_opt  ULONG *pcbWritten)
    {
        return E_NOTIMPL;
    };

    // IUnknown
 STDMETHODIMP_(ULONG) AddRef()
 {
  return InterlockedIncrement(&m_cRef);
 };

 STDMETHODIMP_(ULONG) Release()
 {
  LONG cRef = InterlockedDecrement(&m_cRef);

  if (0 == cRef)
  {
   delete this;
  }

  return cRef;
 };

 STDMETHODIMP QueryInterface(REFIID riid, __deref_out_opt void **ppv)
 {
  HRESULT hr = S_OK;

  if (ppv)
  {
   *ppv = NULL;
  }
  else
  {
   hr = E_INVALIDARG;
  }

  if (S_OK == hr)
  {
   if ((__uuidof(IUnknown) == riid) || (riid == __uuidof(ISequentialStream)))
   {
    AddRef();
    *ppv = (ISequentialStream*)this;
   }
   else
   {
    hr = E_NOINTERFACE;
   }
  }

  return hr;
 };

protected:
 LONG m_cRef;
    void *m_pBuffer;
    size_t m_buffSize;
    size_t m_buffSeekIndex;

    // constructor/deconstructor
 CStringStream(
            __in size_t buffSize,
            __in void *pBuff)
        :
            m_cRef(1), 
            m_pBuffer(pBuff), 
            m_buffSize(buffSize),
            m_buffSeekIndex(0)
    {
    };

    ~CStringStream()
    {
        free(m_pBuffer);
    };
};

Howto Use XmlLite

2009-08-12T15:02:00.000-07:00

I was recently breaking off high-level heavy-weight dependencies on a code I was cleaning up, and I ran into the 500 lb. gorilla that is MSXML6. I found some code that was using it to parse some basic XML strings. MSXML is a full featured XML parser that can do fancy things like schema validation, but it is kind of heavy weight and has high-level dependencies. The downsides of MSXML might be a necessary evil if you need its fancy features, but in many cases we don’t. In my code, I definitely did not. I wanted to gut XML out altogether, but was vetoed. My thoughts turned to MSXML’s handsome and more athletic cousin, XmlLite. XmlLite has very few dependencies and is self-contained in its own library files. Although XmlLite is COM like, it doesn’t even actually have a dependency on COM, so I am liking this guy already. It does need an IStream, or an ISequentialStream, so you will have to create one from some file, or implement the interface yourself. I can provide a sample implementation of that later.

To the code…

Here is a simple quick and dirty code I wrote mainly following the code samples on MSDN. This program takes a filename as a parameter, opens it, and parses the XML printing out the elements. The code I actually wrote looks cleaner, but this will get you going.

MSDN Refrences

MSXML6

XmlLite


/*
 * xml_lite.cpp
 *
 * Description : simple code to show using XML Lite
 */

#include <objbase.h>
#include <XmlLite.h>
#include <shlwapi.h>
#include <stdio.h>

HRESULT WriteAttributes(IXmlReader* pReader)
{
    const WCHAR* pwszPrefix;
    const WCHAR* pwszLocalName;
    const WCHAR* pwszValue;
    HRESULT hr = pReader->MoveToFirstAttribute();

    if (S_FALSE == hr)
        return hr;
    if (S_OK != hr)
    {
        wprintf(L"Error moving to first attribute, error is %08.8lx", hr);
        return -1;
    }
    else
    {
        while (TRUE)
        {
            if (!pReader->IsDefault())
            {
                UINT cwchPrefix;
                if (FAILED(hr = pReader->GetPrefix(&pwszPrefix, &cwchPrefix)))
                {
                    wprintf(L"Error getting prefix, error is %08.8lx", hr);
                    return -1;
                }
                if (FAILED(hr = pReader->GetLocalName(&pwszLocalName, NULL)))
                {
                    wprintf(L"Error getting local name, error is %08.8lx", hr);
                    return -1;
                }
                if (FAILED(hr = pReader->GetValue(&pwszValue, NULL)))
                {
                    wprintf(L"Error getting value, error is %08.8lx", hr);
                    return -1;
                }
                if (cwchPrefix > 0)
                    wprintf(L"Attr: %s:%s=\"%s\" \n", pwszPrefix, pwszLocalName, pwszValue);
                else
                    wprintf(L"Attr: %s=\"%s\" \n", pwszLocalName, pwszValue);
            }

            if (S_OK != pReader->MoveToNextAttribute())
                break;
        }
    }
    return hr;
}

int __cdecl wmain(
        __in int argc, 
        __in_ecount(argc) LPCTSTR argv[])
{
    HRESULT hr = S_OK;
    IStream *pStream = NULL;
    IXmlReader *pReader = NULL;
    UINT cAttribute = 0;

    if (FAILED(hr = SHCreateStreamOnFile(argv[1], STGM_READ, &pStream)))
    {
        wprintf(L"Error creating file reader, error is %08.8lx", hr);
        return hr;
    }

    if (FAILED(hr = CreateXmlReader(__uuidof(IXmlReader), (void**) &pReader, NULL)))
    {
        wprintf(L"error creating xml reader, error is %08.8lx", hr);
        return hr;
    }

    if (FAILED(hr = pReader->SetProperty(XmlReaderProperty_DtdProcessing, DtdProcessing_Prohibit)))
    {
        wprintf(L"Error setting XmlReaderProperty_DtdProcessing, error is %08.8lx", hr);
        return -1;
    }

    if (FAILED(hr = pReader->SetInput(pStream)))
    {
        wprintf(L"Error setting input for reader, error is %08.8lx", hr);
        return -1;
    }

    XmlNodeType nodeType;

    while (S_OK == (hr = pReader->Read(&nodeType)))
    {
        LPCWSTR pwszPrefix = NULL;
        UINT cwchPrefix = 0;
        LPCWSTR pwszLocalName = NULL;
        LPCWSTR pwszValue = NULL;

     switch (nodeType)
  {
  case XmlNodeType_XmlDeclaration:
   wprintf(L"XmlDeclaration\n");
   if (FAILED(hr = WriteAttributes(pReader)))
   {
    wprintf(L"Error writing attributes, error is %08.8lx", hr);
    return -1;
   }
   break;
  case XmlNodeType_Element:
   if (FAILED(hr = pReader->GetPrefix(&pwszPrefix, &cwchPrefix)))
   {
    wprintf(L"Error getting prefix, error is %08.8lx", hr);
    return -1;
   }
   if (FAILED(hr = pReader->GetLocalName(&pwszLocalName, NULL)))
   {
    wprintf(L"Error getting local name, error is %08.8lx", hr);
    return -1;
   }
   if (cwchPrefix > 0)
    wprintf(L"Element: %s:%s\n", pwszPrefix, pwszLocalName);
   else
    wprintf(L"Element: %s\n", pwszLocalName);

   if (FAILED(hr = WriteAttributes(pReader)))
   {
    wprintf(L"Error writing attributes, error is %08.8lx", hr);
    return -1;
   }

   if (pReader->IsEmptyElement() )
    wprintf(L" (empty)");
   break;
  case XmlNodeType_EndElement:
   if (FAILED(hr = pReader->GetPrefix(&pwszPrefix, &cwchPrefix)))
   {
    wprintf(L"Error getting prefix, error is %08.8lx", hr);
    return -1;
   }
   if (FAILED(hr = pReader->GetLocalName(&pwszLocalName, NULL)))
   {
    wprintf(L"Error getting local name, error is %08.8lx", hr);
    return -1;
   }
   if (cwchPrefix > 0)
    wprintf(L"End Element: %s:%s\n", pwszPrefix, pwszLocalName);
   else
    wprintf(L"End Element: %s\n", pwszLocalName);
   break;
        /*
  case XmlNodeType_Text:
  case XmlNodeType_Whitespace:
   if (FAILED(hr = pReader->GetValue(&pwszValue, NULL)))
   {
    wprintf(L"Error getting value, error is %08.8lx", hr);
    return -1;
   }
   wprintf(L"Text: >%s<\n", pwszValue);
   break;
        */
  case XmlNodeType_CDATA:
   if (FAILED(hr = pReader->GetValue(&pwszValue, NULL)))
   {
    wprintf(L"Error getting value, error is %08.8lx", hr);
    return -1;
   }
   wprintf(L"CDATA: %s\n", pwszValue);
   break;
  case XmlNodeType_ProcessingInstruction:
   if (FAILED(hr = pReader->GetLocalName(&pwszLocalName, NULL)))
   {
    wprintf(L"Error getting name, error is %08.8lx", hr);
    return -1;
   }
   if (FAILED(hr = pReader->GetValue(&pwszValue, NULL)))
   {
    wprintf(L"Error getting value, error is %08.8lx", hr);
    return -1;
   }
   wprintf(L"Processing Instruction name:%S value:%S\n", pwszLocalName, pwszValue);
   break;
  case XmlNodeType_Comment:
   if (FAILED(hr = pReader->GetValue(&pwszValue, NULL)))
   {
    wprintf(L"Error getting value, error is %08.8lx", hr);
    return -1;
   }
   wprintf(L"Comment: %s\n", pwszValue);
   break;
  case XmlNodeType_DocumentType:
   wprintf(L"DOCTYPE is not printed\n");
   break;
  }

        /*
        hr = pReader->GetAttributeCount(&cAttribute);

        if (S_OK == hr)
        {
            wprintf(L"num attrubutes %i\n", cAttribute);
        }
        */
    }

    return hr;
}

SetupDi: How To Enumerate Devices Using SetupDiGetClassDevs

2009-08-10T17:29:00.000-07:00

I am back to work from my month off for paternity leave with a fresh new post. This time I going to write a about a topic that is directly related to my job, devices. In particular, we will look at how to use SetupDi to enumerate present devices and print out a few properties. There are a lot of other APIs available in Windows to do device enumeration. Perhaps I will cover them in later posts. As you will see in this post, SetupDi’s interfaces aren’t the most conducive to sexy code, but for better or worse, SetupAPI is the main way to work with devices in Windows. If you have any opinion on what a good device API should look like in Windows, please leave a comment and let me know.

Windows Vista on there is another API that is maybe easier to use for this kind of task, Function Discovery. If you are interested, check it out at:

Function Discovery PnP Enumeration Example

To the code…

This code is pretty basic. We create an HDEVINFO set of all present dev nodes, and step through each dev node printing out a few properties. I haven’t really looked at this sample code recently, so let me know if you see any problems.


#include <windows.h>
#include <setupapi.h>
#include <stdio.h>

void print_property
(
    __in HDEVINFO        hDevInfo,
    __in SP_DEVINFO_DATA DeviceInfoData,
    __in PCWSTR          Label,
    __in DWORD           Property
)
{
    DWORD  DataT;
    LPTSTR buffer     = NULL;
    DWORD  buffersize = 0;
    
    //
    // Call function with null to begin with, 
    // then use the returned buffer size (doubled)
    // to Alloc the buffer. Keep calling until
    // success or an unknown failure.
    //
    //  Double the returned buffersize to correct
    //  for underlying legacy CM functions that 
    //  return an incorrect buffersize value on 
    //  DBCS/MBCS systems.
    // 
    while (!SetupDiGetDeviceRegistryProperty(
                hDevInfo,
                &DeviceInfoData,
                Property,
                &DataT,
                (PBYTE)buffer,
                buffersize,
                &buffersize))
    {
        if (ERROR_INSUFFICIENT_BUFFER == GetLastError())
        {
            // Change the buffer size.
            if (buffer)
            {
                LocalFree(buffer);
            }
            // Double the size to avoid problems on 
            // W2k MBCS systems per KB 888609. 
            buffer = (LPTSTR)LocalAlloc(LPTR, buffersize * 2);
        }
        else
        {
            break;
        }
    }
    
    wprintf(L"%s %s\n",Label, buffer);
    
    if (buffer)
    {
        LocalFree(buffer);
    }
}

//int main(int argc, char *argv[], char *envp[])
int setupdi_version()
{
    HDEVINFO hDevInfo;
    SP_DEVINFO_DATA DeviceInfoData;
    DWORD i;

    // Create a HDEVINFO with all present devices.
    hDevInfo = SetupDiGetClassDevs(
                    NULL,
                    0, // Enumerator
                    0,
                    DIGCF_PRESENT | DIGCF_ALLCLASSES);
    
    if (INVALID_HANDLE_VALUE == hDevInfo)
    {
        // Insert error handling here.
        return 1;
    }
    
    // Enumerate through all devices in Set.

    DeviceInfoData.cbSize = sizeof(SP_DEVINFO_DATA);

    for (i = 0; SetupDiEnumDeviceInfo(hDevInfo, i, &DeviceInfoData); i++)
    {
        LPTSTR buffer     = NULL;
        DWORD  buffersize = 0;

        print_property(hDevInfo, DeviceInfoData, L"Friendly name :", SPDRP_FRIENDLYNAME);

        while (!SetupDiGetDeviceInstanceId(
                    hDevInfo, 
                    &DeviceInfoData, 
                    buffer, 
                    buffersize, 
                    &buffersize))
        {
            if (buffer)
            {
               LocalFree(buffer);
            }
                
            if (ERROR_INSUFFICIENT_BUFFER == GetLastError())
            {
                // Change the buffer size.
                // Double the size to avoid problems on
                // W2k MBCS systems per KB 888609.
                buffer = (LPTSTR)LocalAlloc(LPTR, buffersize * 2);
            }
            else
            {
                wprintf(L"error: could not get device instance id (0x%x)\n", GetLastError());
                break;
            }
        }

        if (buffer)
        {
            wprintf(L"\tDeviceInstanceId : %s\n", buffer);
        }

        print_property(hDevInfo, DeviceInfoData, L"\tClass :", SPDRP_CLASS);
        print_property(hDevInfo, DeviceInfoData, L"\tClass GUID :", SPDRP_CLASSGUID);
    }
    
    
    if (NO_ERROR != GetLastError() && ERROR_NO_MORE_ITEMS != GetLastError())
    {
        // Insert error handling here.
        return 1;
    }
    
    //  Cleanup
    SetupDiDestroyDeviceInfoList(hDevInfo);
    
    return 0;
}

Removing ATL from Your Code

2009-07-10T15:07:00.000-07:00

Removing ATL from Your Code
COM is all over the place if you are doing win32 programming especially in higher level components like the shell. Should we be using COM as a Windows extension model in general is debatable, but we can leave that discussion for a later time. We want to talk about ATL.

The point of ATL is that it is supposed to help you write COM code more quickly by taking care of the tedious parts of COM programming like implementing IUnknown, managing ref counts, and freeing up memory; however, without going into too much gory detail, using ATL adds many high level dependencies not introduced by simple COM programming. If you are writing a higher level COM object that is already depending on things that use ATL or other high level dependencies, then maybe it doesn’t really matter if you are using ATL. Then it is really just a matter of preference. If you want to limit your high level dependencies and lower the level of you COM object, then you should avoid taking a dependency on ATL. Generally this is what you want to do if you want to write a low level system component that others will depend on and write extensions. I fall into this latter category for a component I want to clean up.

Unfortunately you may have some legacy COM code that took a dependency on ATL. If someone wanted depend on your component for use for something low level, then ATL could be a deal breaker. That is if COM already isn’t, but there are ways to make COM lean and mean without even using OLE. Take UMDF for example. This scenario is precisely the situation I am in. After coming back from vacation, I will be spending the next while removing ATL from some COM components. I will write on the things you need to do as I go through and figure out the processes.

Supressing PreFAST Warnings

2009-07-09T15:48:00.000-07:00

PreFAST is a great static analysis tool that can find lots of bugs for you; however, sometimes it can act like an over protective mother. In my last posts I NULL terminated the strings just to make PreFAST happy, but it felt more like a hack. I don't like leaving hacks in my real code. If there is a PreFAST warning that you feel like is unjustified and you would like you code to PreFAST warning free, you can use a handy #pragma trick to tell PreFAST that you know what you are doing and its okay. Keep in mind you don't want to do this very often, because generally PreFAST warnings should be fixed.

In my case, I was mallocing a buffer for a string that was to be read from the registry. PreFAST was warning me that I should NULL terminate the string. In this case RegEnumValue should do that correctly or give me an error. Since this warning is safe to ignore and I don't want to put in a hack just to get rid of the warning I added this #pragma at the line where the warning was:

#pragma prefast(suppress: 26036, "We expect that RegEnumValue will properly NULL terminate ppszKeyValue.")

Reading Registry Values Using RegEnumValue

2009-07-08T18:35:00.000-07:00

This time I am pretending that I don’t know or care what the name of the value is. You can do that by using RegEnumValue. I wrote a function with this prototype:


__checkReturn HRESULT ReadKeyValue(
        __in LPCWSTR pszSubKey,
        __in DWORD dwIndex,
        __deref_out LPWSTR *ppszKeyValue);

Basically you can read regkey values by its index. If you wanted to print out all of the values you could do it like this:


DWORD keyIndex = 0;

do
{
    hr = HRESULT_FROM_WIN32(ReadKeyValue(psCategory, psSubcategory, keyIndex++, &psKeyValue));
    wprintf(L"%s 0x%x\n", psKeyValue, hr);
    if (hr == HRESULT_FROM_WIN32(ERROR_NO_MORE_ITEMS)) wprintf(L"this means there are no more key values\n");
} while (S_OK == hr);

So here is the complete function. If you knew that you were going to read a lot of values, you might want to open the regkey handle beforehand, do all of your reads, and close it when you are done. Other than that this code should look similar to the previous posting.


__checkReturn HRESULT ReadKeyValue(
        __in LPCWSTR pszSubKey,
        __in DWORD dwIndex,
        __deref_out LPWSTR *ppszKeyValue)
{
    HRESULT hr = S_OK;
    LPWSTR pszSubKey = NULL;
    HKEY hKey = NULL;
    LPWSTR pszValueName = NULL;
    DWORD lpcchValueName = 2048; // a reasonable initial buffer size
    DWORD cbDataSize = 4096;
    DWORD type = 0;

    hr = HRESULT_FROM_WIN32(
            RegOpenKeyEx(
                HKEY_LOCAL_MACHINE,
                pszSubKey,
                0,
                KEY_QUERY_VALUE,
                &hKey));

    if (S_OK == hr)
    {
        *ppszKeyValue = (LPWSTR)malloc(cbDataSize);
        pszValueName = (LPWSTR)malloc(lpcchValueName * sizeof(wchar_t));

        if (!*ppszKeyValue || !pszValueName)
        {
            hr = E_OUTOFMEMORY;
        }
        else
        {
            **ppszKeyValue = L'\0';
            *pszValueName = L'\0';
        } 
    }

    while (S_OK == hr && S_OK != (hr = HRESULT_FROM_WIN32(
            RegEnumValue(
                hKey,
                dwIndex,
                pszValueName,
                &lpcchValueName ,
                NULL,
                &type,
                (LPBYTE)*ppszKeyValue,
                &cbDataSize))))
    {
        if (*ppszKeyValue)
        {
            free(*ppszKeyValue);
            *ppszKeyValue = NULL;
        }

        if (pszValueName)
        {
            free(pszValueName);
            pszValueName = NULL;
        }

        if (HRESULT_FROM_WIN32(ERROR_MORE_DATA) == hr)
        {
            hr = S_OK;

            *ppszKeyValue = (LPWSTR)malloc(cbDataSize);
            pszValueName = (LPWSTR)malloc(lpcchValueName * sizeof(wchar_t));

            if (!*ppszKeyValue || !pszValueName)
            {
                hr = E_OUTOFMEMORY;
            }
            else
            {
                **ppszKeyValue = L'\0'; // PreFAST is still whining about this
                *pszValueName = L'\0';
            } 
        }
        else
        {
            break;
        }
    }

    // cleanup
    (void)RegCloseKey(hKey);

    if (pszValueName)
    {
        free(pszValueName);
    }

    if (S_OK != hr && *ppszKeyValue)
    {
        free(*ppszKeyValue);
        *ppszKeyValue = NULL;
    }

    return hr;
}

Posting Source Code in Blogger

2009-07-08T11:14:00.000-07:00

It looks like blogger does not have a good built-in way to post code which is lame since this blog is supposed to be about code. For my last post I used a <pre> tag so the code didn't get foobared. The <pre> tag tells your browser to use a fixed width font and preserve the spacing. I supposed that is the easiest fix, and I will go with it for now.

Googling I found this (santaxhighlighter) other solution that looks more robust. I will do all of the santax highlighting and what not. It looks like it could be nice, but more overhead and annoyance.

I guess I could also use something like Word 2007 to publish to Blogger along with all of its bloated html.

It would be great if Blogger just provided a code mode.

Reading Values From the Registry Using RegGetValue

2009-07-08T10:24:00.000-07:00

I am working on rewriting this section of code that needs to read key values from the registry.

First of all, if you already know what the key value is, you can get directly with RegGetValue.

// you should always use SAL annotations and the free static analysis tool PreFAST
__checkReturn HRESULT ReadKeyValue(
       __in LPCWSTR pszSubKey,
       __deref_out LPWSTR *ppszKeyValue)
{
   HRESULT hr = S_OK;
   DWORD cbDataSize = 4096;

   *ppszKeyValue = NULL;

   // This preallocates a reasonable size buffer for the value I want to read
   // so that 99% of the time we only have to do one trip to the registry
   if (!(*ppszKeyValue = (LPWSTR)malloc(cbDataSize)))
   {
       hr = E_OUTOFMEMORY;
   }
   else
   {
       **ppszKeyValue = L'\0'; // this makes PreFAST happy
   }

   while (S_OK == hr && S_OK != (hr = HRESULT_FROM_WIN32(
           RegGetValue(
               HKEY_LOCAL_MACHINE,
               pszSubKey,
               szRegValue,
               RRF_RT_REG_SZ,
               NULL,                                                                                                 
               *ppszKeyValue,
               &cbDataSize))))
   {
       // if our preallocated buffer is not big enough, we will make it the
       // exact size here and try the registry again
       if (*ppszKeyValue)
       {
           free(*ppszKeyValue);
           *ppszKeyValue = NULL;
       }

       if (HRESULT_FROM_WIN32(ERROR_MORE_DATA) == hr)
       {
           hr = S_OK;

           if(!(*ppszKeyValue = (LPWSTR)malloc(cbDataSize)))
           {
               hr = E_OUTOFMEMORY;
           }
       }
       else
       {
           // this means we hit an unexpected failure
           break;
       }
   }

   // you should always clean up you memory on error states so you don't leak
   if (S_OK != hr && *ppszKeyValue)
   {
       free(*ppszKeyValue);
       *ppszKeyValue = NULL;
   }

   return hr;
}

Welcome to Sam's Code

2009-07-08T10:17:00.000-07:00

I am starting this blog to journal my day to day coding. As I find out how to do new things, I will post them here. I have been doing a lot of device related systems programming on Windows. I will probably post those kind of things first. I also have tons of old stuff for low-level Linux/Unix and GPU programming that may or may not make it on here.