Tuesday, September 8, 2015

WinRT ABI Debugging Workflow

VS can debug a WinRT app's code, but how do you debug the actual ABI?  Here are some handy tricks to deploying and testing the ABI.

Setting Up the Test Box

For my systems programming, I like to have a development machine, and a separate test machine.  This should be obvious, but yes, you can hose your development machine by replacing system binaries with modified ones.  At this point, many of my coworkers simply use VMs.  These days, you can grab a VHD directly off of the build share, so you don't even have to install the OS anymore.  Maybe I am old fashioned, but I actually like to have a physical PC to test on.  Generally it is faster and more reliable.  Furthermore, I have less distractions, so I can debug faster.

What you need to install & configure:
  1. Install a correct build of the OS.
  2. Install WinDBG.  I guess it is part of the WDK?
  3. Copy over the symbols locally.  I think you can also get these from the WDK?  You can also try to rely on the symbol server.
  4. Set you PC to "Developer mode"
    Settings->Update & Security->For developers->Developer mode
  5. Install Remote Tools For VS 2015 : http://www.microsoft.com/en-us/download/details.aspx?id=48155
  6. Run the remote debugging configuration:
    Start -> All Apps -> Visual Studio 2015 -> Remote Debugging Configuration -> Remote debugger

    Click "Configure remote debugging"

  7. Then you will see something like this:



    This means your test box is ready.

Setting Up the Dev Box

To use the fancy new remote app testing feature, you just need to:
  1. Install VS2015.
  2. Install the Windows SDK from the same build installed on your test machine.

Workflow


  1. Dev Box: Write a test app to test the ABI functionality of the ABI.
  2. Dev Box: Build the app in VS2015 for the architecture of the test machine.
  3. Test Box: Run the "Remote debugger" desktop app
  4. Dev Box: Hit the down arrow for start debugging, and select "Remote machine."  If it is the first time you have done this, it will have a dialog to select your test machine.  Select it and hit play.  If you don't see your test machine, see #3.

  5. Test: Validate that your test app is running and working.
  6. Dev: Hit the stop button.

    Why did we bother to start the app to just stop it without any debugging?  It is mainly to just get the app deployed to the test machine.
  7. Test: find the name of the app package for your test app.

    plmdebug /query

    The output should look something like this:
    Package full name: 02985851.Mint.comPersonalFinance_1.2.0.2529_x86__xm34sne40gbwr
    Package state: Unknown
    Package full name: 1ED5AEA5.AngryBirdsSpace_1.6.1.0_x86__p2gbknwb5d8r2
    Package state: Unknown

    In case you were wondering PLM stands for the Process Lifetime Manager.
  8. Test: Then you run:

    plmdebug /enabledebug [package full name here]  "c:\debuggers\windbg.exe"

    This will set the correct image execution options so that windbg will attach to the app container before the app starts running.  Whenever your app starts, you will get windbg window to pop.  This works for everything: foreground apps, background tasks, app activation contracts, etc.
  9. Test: Once, windbg launches, set your break points in your code as you would with a normal Win32 code, and let the debugger go.  
  10. Test: Run through your test app that executes the code you are interested in.



Friday, August 14, 2015

How Can you Find Who is Using What Device Programmatically in Windows?

Well, that is not an easy question to answer.  The short answer is that there is no API to do what you are asking.  In fact the OS isn't really designed to allow that, so some of the information required to figure it out 100% is not preserved in the kernel's object manager.  In other words, you can't, not 100% anyway.  Even Mark Russinovich could figure out how to make a sysinternal tool to answer the question.  With that said, it is possible to figure out quite a lot about who is using what.

First off, people use devices by opening a handle to them using CreateFile.  You typically pass in a device interface path that you discovered using SetupAPI, or one of the other APIs Windows has.  The IO Manager will resolve the device directly, or it also has other branch points where it can involve drivers on the stack to resolve the device.  Ultimately the client gets a handle to the device.  I will leave it at that.  The important thing to know is that devices that are being used have handles backed by objects that the kernel's object manager tracks.

Devices being used will have open handles, so you can look at the handles; however, they don't always tell you enough information of what they are for.  With experience, you can recognize some types of handles, and even figure out what interfaces the match to.  Still, some other handles to device, will not have that information: they can be symlinked, they could just be a named object created directly by the device driver (assuming the set the right permissions, etc.), or a lot of other reasons.

Next, are we talking about user mode or kernel mode?  Drivers in KM can open handles to other devices and they will not be directly visible to a UM process.  You will not be able to see a lot of the references, unless you are discovering them in KM.  If you are talking about a UM tool, then the tool could open other processes in UM and enumerate their handles, assuming it has enough permissions.  It would probably need TCB (trusted computing base) to open them all.  The UM tool will not have visibility into the KM handles.

Do you think you could write such a tool?  It would be a great way to learn how IOMGR works, and PnP.  I could write one, but it still would not be 100% complete.

A good starting place to learn about this is another tool called KD (the Kenrel Debugger).  Dump all of the handles in the system, and go spelunking.  That will give you a lot of experience you would need to write the tool I was talking about.

-Sam

How to Capture Control Events (eg. Ctrl-C) in a C/C++ Windows Console App

I was recently working on a tool of mine that implements a RPC client for my RPC server.  I wanted to let the win32 console app run until the user hits ctrl-c or ctrl-break.  I guess you can also do something like getchar, but I didn't want to.  Also, I could just let the user hit ctrl-c and it would kill the app.  I do use RPC handle rundowns to clean up the RPC server state; however, I wanted to clean up and exit "safely" within the app.

So, without further ado, here is the code for using SetConsoleCtrlHandler in Windows for handling console control signals.

There are a few caveats you need to think about though when implementing control handlers.  For some events, there may be other handlers already registered, so your's may not get called.  This happens, for instance, when using a debugger.  Processes with a window message pump will also get some events through the pump that are not synchronized the process signal handler.

 
/*--
    User mode only.
--*/

#include "precomp.h"

HANDLE g_hControlEvent = NULL;

///// ctl event handling
BOOL WINAPI ControlHandlerCallback(
    _In_ DWORD dwCtrlType)
{
    HRESULT hr = S_OK;
    BOOL bControlEventHandled = FALSE;

    wprintf(L"got event %i\n", dwCtrlType);

    switch (dwCtrlType) {
    case CTRL_C_EVENT:
    case CTRL_BREAK_EVENT:

        if (!SetEvent(g_hControlEvent)) {
            hr = HRESULT_FROM_WIN32(GetLastError());
            NT_ASSERT(hr == S_OK);
        }

        if (S_OK == hr) {
            bControlEventHandled = TRUE;
        }

        break;
    CTRL_CLOSE_EVENT:
    CTRL_LOGOFF_EVENT:
    CTRL_SHUTDOWN_EVENT:
    default:
        break;
    }

    return bControlEventHandled;
}  // ControlHandlerCallback

HRESULT RegisterControlHandler()
{
    HRESULT hr = S_OK;

    g_hControlEvent = NULL;
    g_hControlEvent = CreateEvent(NULL, TRUE, FALSE, NULL);

    if (!g_hControlEvent) {
        hr = HRESULT_FROM_WIN32(GetLastError());
        goto exit;
    }

    if (!SetConsoleCtrlHandler(ControlHandlerCallback, TRUE)) {
        hr = HRESULT_FROM_WIN32(GetLastError());
        goto exit;
    }

    wprintf(L"press ctrl+c to stop\n");

exit:

    return hr;
}  // RegisterControlHandler

HRESULT UnregisterControlHandler()
{
    HRESULT hr = S_OK;

    if (!SetConsoleCtrlHandler(ControlHandlerCallback, FALSE)) {
        hr = HRESULT_FROM_WIN32(GetLastError());
        goto exit;
    }

    if (!CloseHandle(g_hControlEvent)) {
        hr = HRESULT_FROM_WIN32(GetLastError());
    }

    g_hControlEvent = NULL;

exit:

    return hr;
}  // UnregisterControlHandler

HRESULT WaitForControlEvent()
{
    HRESULT hr = S_OK;

    if (!g_hControlEvent) {
        // called before registring
        hr = E_UNEXPECTED;
        goto exit;
    }

    if (WAIT_OBJECT_0 != WaitForSingleObject(g_hControlEvent, INFINITE)) {
        hr = E_FAIL;
        goto exit;
    }

exit:

    return hr;
}  // WaitForControlEvent

Monday, July 6, 2015

MIDL: [ptr] vs [ref] vs [unique] vs [optional]

If you author IDL files for RPC, COM, or WRL, you will probably need to pass arrays.  There are a few parameter attributes you can use for pointers, but it may be unclear which one to use.  You may have seen [ptr], [ref], [unique], and maybe [optional].  Straight off, even though optional seems like a logical choice for emulating SAL's _opt, it is actually a red hearing.  For some inexplicable reason, optional only applies to VARIANT structs according to MSDN.

That leaves us with: ptr, ref, and unique.


In general, I would recommend using ref for const type in parameters, and unique for opt type in parameters.  You should probably stay away from ptr pointers due to the RPC runtime overhead unless you have a good reason to use them.  Also you may experiment with the MIDL compiler and see what SAL annotations are generated with which IDL attributes.  There is a lot more nuance, so dive into the MSDN articles if you need more depth on the topic.

Thursday, March 19, 2015

What Have I Been Working On the Past Year?

Winodws 10. It is interesting to hear my boss' historical perspective. It is good inside info if you don't work at Microsoft.

Monday, March 16, 2015

Making Changes to the Windows Kernel

We all have to modify the Windows kernel from time to time...am I right?  If you are like me, you don't have to do all that often, but when I do, I always seem to forget to do these small things.

The kernel you build must be signed or it will not load.  As a side note, also many core DLLs must be singed like cfgmgr32.dll for instance.  They are checked by smss.exe early in boot, and will cause your system to bugcheck (with 0xc000021a aka STATUS_SYSTEM_PROCESS_TERMINATED) if they are not signed.  Also, how is the build signed?  If it is PRS signed build, you will need to install the test signing certificate to the target if you want to run a test signed kernel.  Make sure these environment variables are set.
set NT_SIGNCODE=1
set NT_SIGNCODE_PH=1

Your kernel and HAL need to match.  This may also be case with other components like ACPI, etc. but these are less likely to cause you problems.  You should just always build replace the kernel and the HAL together.

The are found in c:\windows\system32\ and are called:
x86
ntkrpamp.exe
halmacpi.dll
=or=
AMD64
ntkrnlmp.exe
hal.dll

You can just clobber them, and reboot, but your system will probably just bugcheck.  I would suggest replacing them with alternative names.  Try the following:
reagentc /disable
bcdedit /bootdebug on
bcdedit /set BootStatusPolicy IgnoreAllFailures
bcdedit /set testsigning yes
bcdedit /set kernel mykernel.exe
bcdedit /set hal myhal.dll

Likewise, you should setup up a KD on the target so you can see what bugchecks you are seeing.  Ex:
    bcdedit /debug on
    bcdedit /dbgsettings 1394 channel:1


    But, what if you forgot one of these steps and now your machine is in a bugcheck loop that you can't debug?  You can add them temporarily by pressing F10 to while the Windows boot manager is running, and you can add them in as boot options.

    If you need to change the kernel and the PC will not boot.  You can simply change the kernel or HAL offline using WinPE.  There are lots of ways to get into WinPE, so I won't describe them here.

    Handy tip: here is the command to see what drive is mounted as what in WinPE
    wmic LOGICALDISK LIST BRIEF

    Hopefully this was a handy refresher!

    Tuesday, February 17, 2015

    SAL to SAL2 Porting Guide

    I have written before that you should use SAL in your Windows C/C++ code.  It allows to you do neat things like use Microsoft's static analysis and kill lots of common bugs at compile time.  Whereas C can be ambiguous how say a pointer parameter is used, SAL can make the intended use of parameters and struct members very crisp.  A: if it is hard to figure out what SAL annotation defines your parameter's use in the function, you are probably doing it wrong.  In general, you should make your functions use the parameters in straightforward ways.  B: Likewise, there is always a correct set of SAL annotations for a parameter's behavior, so don't cop out and figure it out.  If you are still confused, refer to A.

    Why port to SAL2?  Because 2 > 1 obviously.  SAL2 does add some new functionality.  You should not be lazy and just use SAL2 in new code, and clean up old code to use SAL2 while you are in there working on it.  Should you go back and fix old code to use SAL2?  It is up to you.  It has some new functionality, however, SAL1 will continue to work just fine.

    Const Protection, Use It

    This isn't SAL, just regular C.  Use const protection.  Parameters that should not be modified should be const protected.  For instance: _In_ const char *p is more correct than _In_ char *p if the p buffer should not be modified.  Please use const correctly for parameters and struct members.

    Here is a table for the basics of SAL2.

    Here is my version with SAL to SAL2 notes.

    Pointer Parameters

    For the annotations in the following table, when a pointer parameter is being annotated, the analyzer reports an error if the pointer is null. This applies to pointers and to any data item that's pointed to.

    SAL 1
    SAL 2
    Description
    __in
    _In_
    Annotates input parameters that are scalars, structures, pointers to structures and the like. Explicitly may be used on simple scalars. The parameter must be valid in pre-state and will not be modified.
    __out
    _Out_
    Annotates output parameters that are scalars, structures, pointers to structures and the like. Do not apply this to an object that cannot return a value—for example, a scalar that's passed by value. The parameter does not have to be valid in pre-state but must be valid in post-state.
    __inout
    _Inout_
    Annotates a parameter that will be changed by the function. It must be valid in both pre-state and post-state, but is assumed to have different values before and after the call. Must apply to a modifiable value.
    __in_z
    _In_z_
    A pointer to a null-terminated string that's used as input. The string must be valid in pre-state. Variants of PSTR, which already have the correct annotations, are preferred.  <= This is a pet annoyance of mine, don't be the guy that does _In_z_ PCWSTR.
    __inout_z
    _Inout_z_
    A pointer to a null-terminated character array that will be modified. It must be valid before and after the call, but the value is assumed to have changed. The null terminator may be moved, but only the elements up to the original null terminator may be accessed.
    __in_ecount(s)
    __in_bcount(s)
    _In_reads_(s)
    _In_reads_bytes_(s)
    A pointer to an array, which is read by the function. The array is of size s elements, all of which must be valid.
    The _bytes_ variant gives the size in bytes instead of elements. Use this only when the size cannot be expressed as elements. For example, char strings would use the _bytes_ variant only if a similar function that uses wchar_t would.
    __in_ecount_z(s) 
    _In_reads_z_(s)
    A pointer to an array that is null-terminated and has a known size. The elements up to the null terminator—or s if there is no null terminator—must be valid in pre-state. If the size is known in bytes, scale s by the element size.

    _In_reads_or_z_(s)
    A pointer to an array that is null-terminated or has a known size, or both. The elements up to the null terminator—or s if there is no null terminator—must be valid in pre-state. If the size is known in bytes, scale s by the element size. (Used for the strn family.)

    _Out_writes_(s)
    _Out_writes_bytes_(s)
    A pointer to an array of s elements (resp. bytes) that will be written by the function. The array elements do not have to be valid in pre-state, and the number of elements that are valid in post-state is unspecified. If there are annotations on the parameter type, they are applied in post-state. For example, consider the following code.
    C++
    typedef _Null_terminated_ wchar_t *PWSTR;
    void MyStringCopy(_Out_writes_ (size) PWSTR p1,
       _In_ size_t size,
       _In_ PWSTR p2);
    In this example, the caller provides a buffer of size elements for p1. MyStringCopy makes some of those elements valid. More importantly, the _Null_terminated_ annotation onPWSTR means that p1 is null-terminated in post-state. In this way, the number of valid elements is still well-defined, but a specific element count is not required.
    The _bytes_ variant gives the size in bytes instead of elements. Use this only when the size cannot be expressed as elements. For example, char strings would use the _bytes_ variant only if a similar function that uses wchar_t would.

    _Out_writes_z_(s)
    A pointer to an array of s elements. The elements do not have to be valid in pre-state. In post-state, the elements up through the null terminator—which must be present—must be valid. If the size is known in bytes, scale s by the element size.

    _Inout_updates_(s)
    _Inout_updates_bytes_(s)
    A pointer to an array, which is both read and written to in the function. It is of size s elements, and valid in pre-state and post-state.
    The _bytes_ variant gives the size in bytes instead of elements. Use this only when the size cannot be expressed as elements. For example, char strings would use the _bytes_ variant only if a similar function that uses wchar_t would.

    _Inout_updates_z_(s)
    A pointer to an array that is null-terminated and has a known size. The elements up through the null terminator—which must be present—must be valid in both pre-state and post-state. The value in the post-state is presumed to be different from the value in the pre-state; this includes the location of the null terminator. If the size is known in bytes, scale s by the element size.

    _Out_writes_to_(s,c)
    _Out_writes_bytes_to_(s,c)
    _Out_writes_all_(s)
    _Out_writes_bytes_all_(s)
    A pointer to an array of s elements. The elements do not have to be valid in pre-state. In post-state, the elements up to the c-th element must be valid. If the size is known in bytes, scale s and c by the element size or use the _bytes_ variant, which is defined as:
    C++
       _Out_writes_to_(_Old_(s), _Old_(s))
       _Out_writes_bytes_to_(_Old_(s), _Old_(s))
    In other words, every element that exists in the buffer up to s in the pre-state is valid in the post-state. For example:
    C++
    void *memcpy(_Out_writes_bytes_all_(s) char *p1,
       _In_reads_bytes_(s) char *p2,
       _In_ int s);
    void * wordcpy(_Out_writes_all_(s) DWORD *p1,
       _In_reads_(s) DWORD *p2,
       _In_ int s);

    _Inout_updates_to_(s,c)
    _Inout_updates_bytes_to_(s,c)
    A pointer to an array, which is both read and written by the function. It is of size s elements, all of which must be valid in pre-state, and c elements must be valid in post-state.
    The _bytes_ variant gives the size in bytes instead of elements. Use this only when the size cannot be expressed as elements. For example, char strings would use the _bytes_ variant only if a similar function that uses wchar_t would.

    _Inout_updates_all_(s)
    _Inout_updates_bytes_all_(s)
    A pointer to an array, which is both read and written by the function of size s elements. Defined as equivalent to:
    C++
       _Inout_updates_to_(_Old_(s), _Old_(s))
       _Inout_updates_bytes_to_(_Old_(s), _Old_(s))
    In other words, every element that exists in the buffer up to s in the pre-state is valid in the pre-state and post-state.
    The _bytes_ variant gives the size in bytes instead of elements. Use this only when the size cannot be expressed as elements. For example, char strings would use the _bytes_ variant only if a similar function that uses wchar_t would.

    _In_reads_to_ptr_(p)
    A pointer to an array for which the expression p – _Curr_ (that is, p minus _Curr_) is defined by the appropriate language standard. The elements prior to p must be valid in pre-state.

    _In_reads_to_ptr_z_(p)
    A pointer to a null-terminated array for which the expression p – _Curr_ (that is, p minus_Curr_) is defined by the appropriate language standard. The elements prior to p must be valid in pre-state.

    _Out_writes_to_ptr_(p)
    A pointer to an array for which the expression p – _Curr_ (that is, p minus _Curr_) is defined by the appropriate language standard. The elements prior to p do not have to be valid in pre-state and must be valid in post-state.

    _Out_writes_to_ptr_z_(p)
    A pointer to a null-terminated array for which the expression p – _Curr_ (that is, p minus_Curr_) is defined by the appropriate language standard. The elements prior to p do not have to be valid in pre-state and must be valid in post-state.