This series is intended to demonstrate and teach operating system development from the ground up.

Introduction

This tutorial covers setting up Microsoft Visual C++ 2005 to work in Kernel Land. This will allow us to use, and completely work in, the nice MSVC++ 2005 IDE.

We will be using the Express Edition. If you have the professional edition, some of the screen shots will be different. Do not worry as most of the options are still the same.

This tutorial will also work fine if you have Visual C++ 2008 or Visual C++ 2010 (with some slight modifications in what options to set).

Getting a high level language to work can be tricky to work and set up with. Getting the build envirement to work itself adds on even more complexity.

The first problem, for example, is that MSVC++ only outputs Win32-compilent PE executables and DLL's. And, at our current stage, we only have a flat, pure binary program loaded at absolute address 1 MB. How do we execute a full blown C++ program, from our flat pure-assembler program?

And yet, that is the beginning of the problems. C++ itself relies on the C++ runtime library. So? The C++ runtime library relies on the operating system. Because we are developing the operating system, C++ has no standard library to work with within our operating system. This means, that we have to work without any standard runtime.

But wait! Remember that, in application software, the runtime needs to initialize everything for our C++ envirement (Such as executing global constructors, providing basic C++ operation support, and executing our main())? Because we have no runtime, we have to do everything ourselves. This produces an interesting chicken and egg scenario: How can we develop a runtime envirement without a runtime for it to work in?

As you can probably see, it can be quite tricky to get C++ to even work properly.

In this tutorial, we are going to set up MSVC++ 2005 for kernel development, and set up the language to work for us. We will also be watching for runtime integrations (i.e, watching where basic C++ relies on the runtime, so we can build our own).

Please note, however, that some C++ features rely on details that we have yet to impliment. These details are fairly advanced. For example, the C++ new and delete operaters require that we have a working memory manager already. As such, this section cannot cover everything just yet. Nontheless, we will set as much things up as possible here.

Ready?

Setting up a new Project

Within the IDE, Select File->New->Project. You should see a nice dialog:

Now, type in the name of the project, and choose the project location. When you are done, select the "OK" button, which should be highlighted.

Your new project should now be created.

The Build Envirement - Project Properties

Because of this, we need to change the configuation settings.

To access Project Properties, Right Click the project name, and select Properties. You should recieve a dialog, simular to the following:

All of the configuation settings are discussed in the same format listed in the above picture (Please look at the left plane).

Project Wide Settings

Also, remember that We are still at Ring 0. As such, we still have control over every little thing. This is important, because if the code the compilier produces does not work with our current operating system setup, the code will triple fault.

This is a problem. Alot of the configuation settings effect the output code. Weather these changes cause a triple fault, or work completely depends on the layout of your operating system, and its configuations. Alot of these changes do not need to be changed. However, because certain configuations may cause a triple fault, this indicates to be extra careful when trying to produce "The fastest and greatest code" from your C++ compilier.

There are, however, some options that REQUIRE changing. These are the options that I will look at here. I will also cover some other options that one may find helpful. I will explain these specific options in depth.

At the end of this section, I will post my current configuation, cool? This way, you can compare your configuations, and learn about setting up MSVC++ 2005.

Configuation Type

C++ Configuation Settings

C/C++ > General

Additional Include Directories

This option allows us to provide our own path for INCLUDE files. This will allow us to use the format when including our own files:

#include <myheader.h>

Debug Information Format

Although we are using MSVC++, we Cannot use it's debugging features. Debuggers require a runtime envirement to run in (and to hook) with an application. Because we do not have a runtime envirement, you should Disable this.

Normally, Adding debuggng information will not cause any problems. However, because we are unable to use the debugger right now, there is no reason to generate debug info.

Warning Level

Operating System code can get very complex. It is important that we can track even the slightest of potential problems. Because of this, I recommend to Set this to the highest level.

C/C++ > Optomization

This can be set to any option. If a specific setting causes the code to crash, disassemble the code and try to find out why. All source code in this series have been tested with all optomization levels to work.

Omit Frame Pointers

This is not required; but setting this option frees up ESP so we may use it.

C/C++ > Preprocessor

Throughout the source code, I will be hiding all x86 architecture dependencies behind a special preprocessor constant. This makes it easier to port non-portable code to other architectures. This constant is ARCH_X86. This can be #define'd, but putting ARCH_X86 here is easier ;)

Ignore Standard Include Path

Remember we do not have a standard library anymore? :)

C/C++ > Code Generation

Enable C++ Exceptions

This requires runtime support, which we do not have. I plan on implimenting exception handling soon, though. Until then, this should be set to No.

Struct Member Alignment

While writing the Kernel, we will be using alot of classes and structures. Most of these must be byte aligned. Most compiliers add extra padding (for speed) to these structures, which will throw off the alignment that we need. Because of this, set this to 1 Byte (/Zp1)

Buffer Security Check

When enabled, MSVC++ adds extra code to test for possible buffer under and overruns. This relies on the MSVC++ runtime, which we cannot use. Because of this, we cannot use this. Set this to No (/GS-)

C/C++ > Language

Enable Run-Time Type Info

Run Time Type Info (RTTI) requires runtime support. Because we are disabling the runtime, we cannot use this. Set to No (/GR-)

C/C++ > Advanced

C/C++ > Command Line

/O2 /Oy /I "..\Include\\" /D "ARCH_X86" /X /FD /MT /Zp1 /GS- /GR- /FAs /Fa"Debug\\"
/Fo"Debug\\" /Fd"Debug\vc80.pdb" /W4 /nologo /c /Gd /TP /errorReport:prompt

Linker Configuation Settings

call _main   ; call C++ main() routine

Linker > General

Linker > Input

By default, MSVC++ automatically links in a number of its libraries, including kernel32.lib, user32.lib, gdi32.lib, winspool.lib and more. Because we never use them, they will not cause any problems. They will add extra uneeded bloat to your kernel, though. Setting this to $(NOINHERIT) will fix this so they will not be linked.

Ignore Default Libraries

There is no standard library, so set this to Yes (/NODEFAULTLIB)

Linker > Debugging

• Set Generate Map File to Yes (/MAP)
• Set Map File Name to the name of the mapfile to generate.
• Hit Apply

Linker > System

SubSystem

This value is stored within the program file. It tells the operating system how to run the application. Because this is a driver application, set this to Native (/SUBSYSTEM:NATIVE).

Driver

This option insures to build this program as a kernel-level driver. This automatically envokes the /FIXED:NO option (instead of the standard /FIXED option), which generates a relocation section, instead of a fixed base address. Because we are developing a driver application, set this option to Driver (/DRIVER)).

Linker > Optimization

Setting Enable COMDAT folding to Remove redundent COMDATS will also reduce the size of the Kernel, and the number of redundent COMDATS.

Linker > Advanced

Entry Point

This should be set to the entry point of your kernel. In our system, this will be kernel_entry.

Base Address

This is the base address that the image will be loaded to. Remember that the kernel is loaded to 1MB? Because of this, this should be set to 0x100000.

Fixed Base Address

Linker > Command Line

Additional options

Add /ALIGN:512 to the Additional options text box. This is required to insure proper section alignment. Not doing so will cause problems executing the kernel or triple faults.

The command line

Thats all! Compare your command line with the following. You may have additional options depending on your envirement setup.

/OUT:"A:\KRNL32.EXE" /INCREMENTAL:NO /NOLOGO /LIBPATH:"..\Lib\\" /MANIFEST:NO
/NODEFAULTLIB /MAP:"Kernel.map" /SUBSYSTEM:NATIVE /DRIVER /OPT:REF /OPT:ICF /ENTRY:"kernel_entry"
/BASE:"0x100000" /FIXED:No /ERRORREPORT:PROMPT

Executing the PE Kernel

The problem, however, is that MSVC++ can only output COFF and PE file formats. Because of this, we have to find a way of parsing it from within our Stage 2 bootloader.

Because I do not plan on describing the PE format in detail yet, I will first describe its basic format, and how the code works. Lets take a look!

File Format

Understanding how to parse file formats are very important. Remember that the FIRST byte within the FIRST structure actually represents the FIRST byte from where it is loaded at in memory.

_IMAGE_DOS_HEADER

typedef struct _IMAGE_DOS_HEADER {  // DOS .EXE header
    USHORT e_magic;         // Magic number (Should be MZ)
    USHORT e_cblp;          // Bytes on last page of file
    USHORT e_cp;            // Pages in file
    USHORT e_crlc;          // Relocations
    USHORT e_cparhdr;       // Size of header in paragraphs
    USHORT e_minalloc;      // Minimum extra paragraphs needed
    USHORT e_maxalloc;      // Maximum extra paragraphs needed
    USHORT e_ss;            // Initial (relative) SS value
    USHORT e_sp;            // Initial SP value
    USHORT e_csum;          // Checksum
    USHORT e_ip;            // Initial IP value
    USHORT e_cs;            // Initial (relative) CS value
    USHORT e_lfarlc;        // File address of relocation table
    USHORT e_ovno;          // Overlay number
    USHORT e_res[4];        // Reserved words
    USHORT e_oemid;         // OEM identifier (for e_oeminfo)
    USHORT e_oeminfo;       // OEM information; e_oemid specific
    USHORT e_res2[10];      // Reserved words
    LONG   e_lfanew;        // File address of new exe header
} IMAGE_DOS_HEADER, *PIMAGE_DOS_HEADER;

The address of the _IMAGE_FILE_HEADER structure is inside of the e_lfanew member of the _IMAGE_DOS_HEADER. So, in order to access this member, we refrence the byte offset from where it is loaded in memory:

mov    ebx, [IMAGE_PMODE_BASE+60]	; e_lfanew is a 4 byte offset address of the PE header; it is 60th byte. Get it
add    ebx, IMAGE_PMODE_BASE 		; add base

Real Mode DOS Stub Program

It is possible to change the program that is executed. This allows us to embed our own program to execute, instead of the dull default one. We do this is MSVC++ using the STUB command line option. For example:

/STUB=myprog.exe

Because our kernel is an EXE file, it is possible for users to double-click and attempt to run it from Windows. Instead, this DOS stub program will run instead. Cool, huh?

Anywhoo...Because the size of this program is not constant, we need to jump over it to the next section--the _IMAGE_FILE_HEADER. This is why we needed to get the location of _IMAGE_FILE_HEADER from the _IMAGE_DOS_HEADER struct.

_IMAGE_FILE_HEADER

typedef struct _IMAGE_FILE_HEADER {
    USHORT  Machine;
    USHORT  NumberOfSections;
    ULONG   TimeDateStamp;
    ULONG   PointerToSymbolTable;
    ULONG   NumberOfSymbols;
    USHORT  SizeOfOptionalHeader;
    USHORT  Characteristics;
} IMAGE_FILE_HEADER, *PIMAGE_FILE_HEADER;

mov    ebx, [IMAGE_PMODE_BASE+60]	; e_lfanew is a 4 byte offset address of the PE header; it is 60th byte. Get it
add    ebx, IMAGE_PMODE_BASE 		; add base
 
; EBX now points to beginning of _IMAGE_FILE_HEADER. Jump over it to the next section (_IMAGE_OPTIONAL_HEADER)
add	ebx, 24

_IMAGE_OPTIONAL_HEADER

struct _IMAGE_OPTIONAL_HEADER {
    //
    // Standard fields.
    //
    USHORT  Magic;
    UCHAR   MajorLinkerVersion;
    UCHAR   MinorLinkerVersion;
    ULONG   SizeOfCode;
    ULONG   SizeOfInitializedData;
    ULONG   SizeOfUninitializedData;
    ULONG   AddressOfEntryPoint;			<< IMPORTANT!
    ULONG   BaseOfCode;
    ULONG   BaseOfData;
    //
    // NT additional fields.
    //
    ULONG   ImageBase;
    ULONG   SectionAlignment;
    ULONG   FileAlignment;
    USHORT  MajorOperatingSystemVersion;
    USHORT  MinorOperatingSystemVersion;
    USHORT  MajorImageVersion;
    USHORT  MinorImageVersion;
    USHORT  MajorSubsystemVersion;
    USHORT  MinorSubsystemVersion;
    ULONG   Reserved1;
    ULONG   SizeOfImage;
    ULONG   SizeOfHeaders;
    ULONG   CheckSum;
    USHORT  Subsystem;
    USHORT  DllCharacteristics;
    ULONG   SizeOfStackReserve;
    ULONG   SizeOfStackCommit;
    ULONG   SizeOfHeapReserve;
    ULONG   SizeOfHeapCommit;
    ULONG   LoaderFlags;
    ULONG   NumberOfRvaAndSizes;
    IMAGE_DATA_DIRECTORY DataDirectory[IMAGE_NUMBEROF_DIRECTORY_ENTRIES];
} IMAGE_OPTIONAL_HEADER, *PIMAGE_OPTIONAL_HEADER;

The important member here is AddressOfEntryPoint which contains...erm... The address of the entry point routine. For example...main(), mainCRTStartup(), whatever suits your needs.

Knowing that EBX points to the beginning of this structure, all we now need to do is refrence EBX+AddressOfEntryPoint. Read from that location, and we have the beginning address of the starting routine to call. After we get this address, all we need to do is far jump to that location, and we effectivly call our C++ entry point!

Putting it together

Remember that the image is loaded at IMAGE_PMODE_BASE:

	mov	ebx, [IMAGE_PMODE_BASE+60]	; e_lfanew is 60th byte. Get it
	add	ebx, IMAGE_PMODE_BASE		; Add base address. EBX now points to file sig (PE00)

	; jump over to optional header (Although it isnt optional o.0 )
 
	add	ebx, 24
	mov	eax, [ebx]		; _IMAGE_FILE_HEADER is 20 bytes + size of sig (4 bytes)
	add	ebx, 16			; address of entry point is now in ebx

Now, EBX contains the address of the entry point routine. Before calling it, however, we need to add the image base address to the entry point address. That is, the entry point address is just an offset.

Looking back at the _IMAGE_OPTIONAL_HEADER structure, we can see the ImageBase member. This is 12 bytes (A ULONG is 4 bytes) from AddressOfEntryPoint. Knowing EBX already points to AddressOfEntryPoint, this is very easy:

	mov	ebp, dword [ebx]	; store entry point address
	add	ebx, 12			; ImageBase member is 12 bytes from AddressOfEntryPoint member
	mov	eax, dword [ebx]	; gets image base
	add	ebp, eax		; add image base to entry point address

	call	ebp	; Execute Kernel

Developing a C++ Runtime Envirement for MSVC++

Because of this, We have to create our own C++ runtime code. This can get tricky. Alot of C++ features requires the use of a runtime. However, because we have disabled the runtime, the compilier will generate interesting errors when using these features. Other times are simply unpredictable, and may cause a triple fault.

Lets look at this, for a moment. In applications, what calls main()? The runtime library. What calls and initialzes all global objects? The runtime library. What provides certain keyword supports that which tie with the system (such as new and delete)? The runtime library. What sets up the initil stack information? Again: The runtime library.

Not defining a runtime library can cause unpredictable results. For one, global and static objects will never be initialized. Another problem is that the use of certain keywords are unpredictable. Global and static objects will never be deallocated. Also, the compilier relies on certain routines--useually defined by the standard runtime. Defining and calling virtual functions may become unpredictable. The calling of pure virtual routines will immediately crash. And, say goodby to new, delete, typeid, and exceptions.

To make a story short: Creating a small C++ runtime is essental to get C++, the language itself, to even work properly for us.

Global Operators

void* __cdecl ::operator new (unsigned int size) { return 0; }
void* __cdecl operator new[] (unsigned int size) { return 0; }
void __cdecl ::operator delete (void * p) {}
void __cdecl operator delete[] (void * p) { }

Pure virtual function call handler

It is not possible to call a pure virtual function directly through normal means. Calling a pure virtual function will result into undefined behavior, because that function does not actually exist--It was never defined.

If a pure virtual function has some how been called, the compilier attempts to use _purecall() as the call handler. If this does not exist, the result is unpredictable--That is, a triple fault.

Because of this, our C++ runtime will need to define it:

int __cdecl ::_purecall() { for (;;); return 0; }

Floating Point Support

There is nothing wrong with this...Just like with using the new and delete operators without them being defined. Simulariy, MSVC++ needs some routines defined for working with floating point math.

_fltused

extern "C" int _fltused = 1;

_ftol2_sse()

extern "C" long __declspec (naked) _ftol2_sse() {
 
	int a;
#ifdef ARCH_X86
	_asm	fistp 	[a]
	_asm	mov	ebx, a
	_asm	ret
#endif
}

Other routines

Initializing globals and static data

To do this, we have to watch exactally how MSVC++ handles the constructors (ctors).

MSVC++ uses a special section (Simular to .data, .bss, .text, etc.) within the final binary image for ctors. When the MSVC++ compilier finds an object that must be executed by the startup code, it places a Dynamic initializer inside of this section. This means that -- for every dynamic initializer that needs to be executed at startup, they can all be found by looking inside of this special section.

This section is the .CRT section. These dynamic initializers are a an array of 4 byte function pointers, which are stored within .CRT So, if we can find a way of parsing this section, we can call each function pointer that MSVC++ set for us to call each routine that needs to be called at startup.

We cannot do this with C++ alone, however, as these section names are very MSVC++ specific. Also, as we are building without any runtime, the .CRT section is currently nonexistant. We have to add this section ourselves. The only way of doing this is using the preprocessor.

Naming Conventions

Actually, these section names do have a purpose. the section names are composed of two parts, seperated by the dollar sign "$". The first part is used as the base section name. The second part indicates where it is located at in the final image.

That is, we can think of a section name having this format:

.section_name$location_name

The section name can be .code, .data, .bss, .CRT, or any other section name. The location_name is an alpabitic name that represents where we are in the section. For an example, in .CRT$XCA, .CRT is the section name, XCA is its location in that section. It does not matter what this location name is; the important thing to remember is that it is in alphabitical order.

Here is an example:

	.CRT$XCA
	.CRT$XCU
	.CRT$XCZ

Lets look at this anoher way, by mixing these up...

	.CRT$XCZ  
	.CRT$XCA
	.CRT$XCU

We will be able to see these sections inside of the linker map, when we set them up.

Creating new segment names

This directive takes the form:

	#pragma data_seg( ["section-name"[, "section-class"] ] )

The important part, thus, is the first parameter--"section-name", which gives the new section name a...er...name:

#pragma data_seg(".CRT$XCA")
 
// All varables allocated here are now placed within the .CRT$XCA section, rather then .data section

//! Select the default data segment again (.data) for the rest of the unit
#pragma data_seg()

Merging Sections

We can fix this by combining the two sections together, which will insure both sections have read and write abilities:

//! Now, move the CRT data into .data section so we can read/write to it
#pragma comment(linker, "/merge:.CRT=.data")

Initializing globals - Setup

//! Function pointer typedef for less typing
typedef void (__cdecl *_PVFV)(void);

/**
*	MSVC++ creates dynamic initializers and deallocaters, which help us in calling the routines.
*	The compilier and linker bind all dynamic initializers into a function pointer table inside a
*	section called .CRT$XCU.
*/
 
// Standard C++ Runtime (STD CRT) __xc_a points to beginning of initializer table
#pragma data_seg(".CRT$XCA")
_PVFV __xc_a[] = { NULL };

__xc_a is a standard MSVC++ CRT name used as a pointer to the beginning of the initializer list, stored in .CRT$XCU

//! .CRT$XCU is located here.

//! Standard C++ Runtime (STD CRT) __xc_z points to end of initializer table
#pragma data_seg(".CRT$XCZ")
_PVFV __xc_z[] = { NULL };

//! Select the default data segment again (.data) for the rest of the unit
#pragma data_seg()

//! Now, move the CRT data into .data section so we can read/write to it
#pragma comment(linker, "/merge:.CRT=.data")

void __cdecl _initterm ( _PVFV * pfbegin, _PVFV * pfend )
{
    // Go through each initializer
    while ( pfbegin < pfend )
    {
      // Execute the global initializer
      if ( *pfbegin != NULL )
            (**pfbegin) ();
 
	// Go to next initializer inside the initializer table
        ++pfbegin;
    }
}
 
// This initializes all global initializer routines:
_initterm(__xc_a, __xc_z);

Cleaning up the envirement

The good news is that this is much easier to work with then the initializer routines. All we need to define a location to store an array of global deinitializer function pointers at some location in memory:

//! function pointer table to global deinitializer table
static _PVFV * pf_atexitlist = 0;
 
//! Maximum entries allowed in table
static unsigned max_atexitlist_entries = 32;
 
//! Current amount of entries in table
static unsigned cur_atexitlist_entries = 0;

We define where these arrays is located. MSVC++ adds deintializer code for each global object, that adds a function pointer to the global deinitializer array. It does this by calling a specially defined routine, atexit ().

Note: MSVC++ requires this routine. Not defining this routine will result in errors when defining a dtor of any kind.

The actual routine is simple. Remember that, for each global object, MSVC++ embeds code that will call this routine. The dtor as an object is passed into this routine as a paramater. Because of this, all we need to do is to add it to the end of our dtor array:

//! For every global object created, MSVC++ calls this routine with a function ptr to each dtor
int __cdecl atexit(_PVFV fn)
{
	//! Insure we have enough free space
	if (cur_atexitlist_entries>=max_atexitlist_entries)
		return 1;
	else {
 
		// Add the exit routine
		*(pf_atexitlist++) = fn;
		cur_atexitlist_entries++;
	}
 
	return 0;
}

void __cdecl _atexit_init(void)
{
    max_atexitlist_entries = 32;
 
	// Warning: Normally, the STDC will dynamically allocate this. Because we have no memory manager, just choose
	// a base address that you will never use for now
    pf_atexitlist = (_PVFV *)0x500000;
}

There are alot of possibly new concepts for our readers, however, so all of this may be better in an example demo.

The Entry Point

void _cdecl kernel_entry () {

#ifdef ARCH_X86
	_asm {
		cli					// clear interrupts--Do not enable them yet
		mov ax, 10h				// offset 0x10 in gdt for data selector, remember?
		mov ds, ax
		mov es, ax
		mov fs, ax
		mov gs, ax
		mov ss, ax				// Set up base stack
		mov esp, 0x90000

		mov ebp, esp
		push ebp
	}
#endif

After calling main(), just halt the system, to insure we don't return (As there is nowhere to return to.)

	//! Execute global constructors
	InitializeConstructors();
 
	//! Call kernel entry point
	main ();
 
	//! Cleanup all dynamic dtors
	Exit ();
 
#ifdef ARCH_X86
	_asm cli
#endif
	for (;;);
}

Demo

This demo loads and executes a 32 bit Kernel written in MSVC++ 2005. It also includes all of the source code in this tutorial, as well.

Conclusion

Yey! Alot of concepts in this tutorial is farily simple, isn't it? We covered setting up MSVC++ 2005 so that we can use the compilier for use in operating system Kernels. We also looked at creating a basic C++ runtime envirement, ctor and dtor calling, virtual function handling, and global operators.

In the next few tutorials, we are looking at creating envirements for different compiliers. This tutorial has covered setting up MSVC++ 2005.

This tutorial was hard to write--and I am yet to finish it. There are simply so many options that MSVC++ has, and describing these options in detail will take a long time. I wanted to find a way of combining context, not just a "do this do that" option setting list. I am still deciding on a format style for that. I hope I did Okay :)

Until next time,

~Mike
BrokenThorn Entertainment. Currently developing DoE and the Neptune Operating System

Questions or comments? Feel free to Contact me.

Would you like to contribute and help improve the articles? If so, please let me know!