This series is intended to demonstrate and teach operating system development from the ground up.

Introduction

Welcome! :)

In the previus tutorial, we looked at how to switch the processor into a 32 bit mode. We also learned how we can access up to 4 GB of memory. This is great--but, how?

We have not talked about the protected mode memory map, and I have failed to mention the little "0x8:Stage3" far jump from the previous tutorial. That is, where did I get 0x8 from?

Also, remember that the PC boots into real mode, which has the limitation of 16 bit registers. And, hence, 16 bit segment addressing. This limits the amount of memory you can access in real mode. Because of this, we still cannot access even up to 1 GB of memory yet. Heck, we cannot even go passed the 1 MB barrier yet! What to do? We have to enable the 20th address line. This will require direct hardware programming, so we will talk about that as well.

So, This is what's on the menu:

• Direct Hardware Programming - Theory
• Direct Hardware Programming and Controllers
• Keyboard Controller Programming - Basics
• Enabling A20
• Pizza :)

Do to the use of high level languages, like C, being able to access more then 1 MB of memory can be critical. Because of this, enabling A20 (Address line 20) will be important!

Note: Remember that we cannot access over 1 MB yet! Doing so will cause a triple fault.

Also, because we are going to go over direct hardware programming, this tutorial will be a little more complicated then previous ones. Don't worry - You will get more expeirence with direct hardware programming later when we develop device drivers for the Kernel.

Ready?

Get Ready

Every single controller must be programmed a special way in order to work correctly. For example, to write (or read) a hard drive, you must first determin if it is an IDE or SCSI drive. Then, you have to determin the drive number it is, and program it using either the IDE Controller or SCSI Controller, which control the IDE and SCSI connections, respecitvley. Both of these controllers are different.

To add more complexity, a "Sector" might not be 512 bytes. Hence, "Reading and writing sectors" is vauge.

Then comes memory management and fragmentation. This is where Paging, Virtual Address Space, and the Memory Management Unit (MMU) comes into play.

Reading and writing any drive is very different then any other drive. This is even true at the bootsector level. The typical format and file system is different between media, so code that boots from a FAT12 floppy will Not work to boot a CDFS Filesystem CD ROM. By Abstracting hardware specific (And low level code), we can make most of the code, however, work for these devices.

When we say "Write a file to hard drive", we normally don't want to define what a "file" is, because we shouldn't. We shouldn't have to worry about what controller to read to/from, nor the exact location on disk. This is why abstraction is *very* important!

Everything here is primairly for protected mode (i.e., it is 32 bit code), although it will work in real mode as well. Because of this, remember the rule of protected mode:

• No interrupts are avilable! The use of any interrupt will cause a triple fault

Kernel Debugging

Debuggers in High Level Languages

What this means is that we need a debugger that could work and display memory directly. Bochs has a debugger just for us.

Bochs Debugger

Bochs debugger and display window will now appear, and you should see the line:

  
	[0x000ffff0] f000:fff0 (unk. ctxt): jmp f000:e05b         ; ea5be000f0
        < bochs:1> _
	

The first line is the important line. It tells you the current instruction, absolute address, and seg:offset address. It also gives you the machine language Operation Code (Opcode) equivelant.

HELP command

BREAK command

  
	[0x000ffff0] f000:fff0 (unk. ctxt): jmp f000:e05b         ; ea5be000f0   < Where BIOS is at
        < bochs:1> b 0x7c00                                                      < Sets the breakpoint to 0x7c00:0
	< bochs:2> c                                                             < Continue executon until a breakpoint is reached
	< 0> Breakpoint 1, 0x7c00 in ?? < >                                      < Our breakpoint is hit
        Next at t=834339
        < 0> [0x00007c00] 0000:7c00 (unk. ctxt): jmp 7cb5         ; e9b200       < We are at our bootloaders first instruction (Which is 'jump main')
	

Knowing that the bootloader loads stage 2 at 0x500, lets break to it:

  
	< bochs:3> b 0x500
        < bochs:4> c
	< 0> Breakpoint 2, 0x500 in ?? < >
	Next at t=934606
	<0> [0x000000500] 0050:0000 (ink. ctxt): jmp 00a0        ; e99d00
        < bochs:5> _
	

Single Step

  
	< bochs:6> s
	Next at t=934607
	<0> [0x0000005a0] 0050:00a0 (ink. ctxt): cli             ; fa
	< bochs:7> s
	Next at t=934608
	<0> [0x0000005a1] 0050:00a1 (ink. ctxt): xor AX, AX      ; 31c0
	< bochs:8> _
	

dump_cpu

print_stack

Conclusion

Direct Hardware Programming - Theory

"Direct hardware programming" simply referrs to communicating directly (and controlling) individual chips. As long as these chips are programmable (In some way), we can control them.

In Tutorial 7, we took a very detailed look at how the system works. We also talked about how software ports work, port mapping, the IN and OUT instructions, and I gave a huge table with common port mappings on x86 architectures.

Remember that, whenever the processor recieves an IN or OUT instruction, It enables the I/O Access Line on the Control Bus, which, of course, is part of the System Bus, in the Motherboards North Bridge. Because the system bus is connected to both the Memory Controller and I/O Controller, both controllers listen for specific addresses and activated lines in the control bus. If the I/O Access line is set (Electricity runs through it--which means it is active (1), The I/O controller takes the address.

The I/O Controller then gives the port address to every other device, and awaits a signal from a controller chip (meaning that it belongs to some device--so give whatever data to that device). If no controller chip responds, and the port address is set back, it is ignored.

This is how port mapping works. (Please see tutorial 7 for more detail.)

Also, remember that a single controller chip may be assigned a range of port addresses. Port addresses are assigned by the BIOS POST, even before the BIOS is loaded and executed. Why? Alot of devices need different types of information. Some ports may represent "registers", while others may be "data" or "ready" ports. It's ugly, I know. But it gets worse. On different systems, port addresses may vary widely. Because x86 architectures are backword compatible, basic devices (Such as keyboards and mice) are useually always the same address. More complex devices, however, may not be.

Direct Hardware Programming and Controllers

Many PCs are based off of the early Intel 8042 Microcontroller chip. This controller chip is either embedded as an IC (Intergrated Circuit) chip, or directly in the motherboard. It is useually locaed in the South Bridge.

This microcontroller communcates through a cord connecting to your keyboard, to another microcontroller chip in your keyboard.

When you press a key on the keyboard, It presses down on a rubber dome setting beneath the key itself. On the underside of the rubber dome is a conductive contact that, when pressed down, comes in contact with two conductive contacts on the keyboard circuit. Because of this, current can flow through. Each key is connected by a pair of electrical lines. As each signal changes (Depending on weather keys are pressed), a make code is generated (From the series of lines). This make code is sent to the microcontroller chip inside of the keyboard, and sent through the cord connecting to the computer hardware port. It is sent through as a series of on and off electrical pusles. Depending on the clock cycles, each pulse can be converted to a series of bits representing a bit pattern.

We are on the motherboard. This series of bits goes through the south bridge as electrical signals, all the way to the 8042 microcontroller. This microcontroller decodes the make code into a scan code, and stores it within an internal register. That is, our buffer. The internal registers can be an EEPROM chip, or simular, so we can electrically overwrite the data whenever we want.

When booting, the BIOS POST assigns each device (Through the I/O Controller) port addresses. It does this by quering the devices. In the useual case, the BIOS POST sets this internal register at port address 0x60. This means, Whenever we refrence port 0x60, we are requesting to read from this internal register.

You know the rest of the story reguarding port mapping, and IN/ OUT instructions, so lets read from that register:

  
		in al, 0x60		; get byte from 8042 microcontroller input register
	

You might be wondering why you have to communicate to the keyboard controller to enable A20. We will look at this next.

Gate A20 - Theory

To fix this problem, Intel put a logic gate on the 20th address line between the processor and system bus. This logic gate got names Gate A20, as it can be enabled and disabled. For older programs, it an be disabled for programs that rely on the wrap wround, and enabled for newer programs.

When booting, the BIOS enables A20 when counting and testing memory, and then disables it again before giving our operating system control.

There are alot of ways to enable A20. By enabling the A20 gate, we have access to all 32 lines on the address bus, and hence, can refrence 32 bit addresses, or up to 0xFFFFFFFF - 4 GB of memory.

The A20 Gate is connected to the 8042 Keyboard Controller. This allows us to go through the keyboard controller to enable and disable this logic gate. Rather then having another controller, Intel allowed access to it using another controller. This gives us a way to control the gate through software. The keyboard controller happens to be the lucky controller. Hence, and alas, in order to enable A20, we have to talk to the 8042 Keyboard Microcontroller.

Keyboard Controller Programming - Basics

What we will look at are the important keyboard controller commands and registers that we need to use to enable A20.

Keyboard Controller - Status Register

The status register is an 8 bit value that represents the current state of the keyboard. It is useually mapped to port address 0x60. (Remember The I/O Port table from Tutorial 7!)

The 8 bit value looks like this:

• Bit 0: Output Buffer status (Cleard when port 0x60 is read from)
• 0: Output buffer empty, dont read yet
• 1: Output buffer full, please read me :)
• Bit 1: Input Buffer Status
• 0: Input buffer empty, can be written
• 1: Input buffer full, dont write yet
• Bit 2: System flag
• 0: Set after power on reset
• 1: Set after successfull completion of the keyboard controllers self-test (Basic Assurance Test, BAT)
• Bit 3: Command Data
• 0: Last write to input buffer was data (via port 0x60)
• 1: Last write to input buffer was a command (via port 0x64)
• Bit 4: Keyboard Locked
• 0: Locked
• 1: Not locked
• Bit 5: Auxiliary Output buffer full
• PS/2 Systems:
• 0: Determins if read from port 0x60 is valid If valid, 0=Keyboard data
• 1: Mouse data, only if you can read from port 0x60
• AT Systems:
• 0: OK flag
• 1: Timeout on transmission from keyboard controller to keyboard. This may indicate no keyboard is present.
• Bit 6: Timeout
• 0: OK flag
• 1: Timeout
• PS/2:
• General Timeout
• AT:
• Timeout on transmission from keyboard to keyboard controller. Possibly parity error (In which case both bits 6 and 7 are set)
• Bit 7: Parity error
• 0: OK flag, no error
• 1: Parity error with last byte

Looks look closer.

Waiting for commands

To wait for the controller to finish with the command, just check if the input buffer is empty at port 0x64. Simple enough. Looking back at the table, you should notice Bit 1: Input Buffer Status. If this bit is set, the input buffer still contains our command. If it is cleared, Our command is completed, so we can send another command.

So, to wait for the controller:

  
		; get current controller status

		KbrdCntrlGetPortStat:
			xor	ax, ax			; clear AX
			in	al, 0x60		; read from kbrd contoller status register

		; 0x64 cleared? Wait until last command completes

		KbrdCntrlWait:
			xor	ax, ax
			in	al, 0x64
			bt	ax, 0
			jc	KbrdCntrlWait

		; send new command
	

Keyboard Controller - Command Register

The commands are:

• Command: 0xDD: Disable A20 Address Line
• Command: 0xDF: Enable A20 Address Line

Lets see this method in action, shall we?

  

		; Wait for keyboard controller to send next command byte

		KbrdCntrlWait:
			xor	ax, ax		; clear AX
			in	al, 0x64	; Get port status--is 0x64 clear?
			bt	ax, 1
			jc	KbrdCntrlWait

		; Enable A20. Note: May not work on all keyboards!

		EnableA20:
			mov	al, 0xDF	; Command 0xDF: Enable A20
			out	0x64, al	; Send command!
	

Keyboard Controller - Command Byte

Because the command byte is just a byte, lets look at it now:

• Bit 0: Keyboard Interrupt Enable
• 0: Disable
• 1: Send Interrupt Request (IRQ1) when output buffer is full
• Bit 1: Mouse Interrupt Enable
• ISA Systems: Unused, always 0.
• 0: Disable
• 1: Send Interrupt Request (IRQ12) when output buffer is full
• Bit 2: Shown in bit 2 of status register
• 0: Cold reboot
• 1: Warm Reboot (Done execute BAT again)
• Bit 3: Keyboard Lock Ignore
• PS/2 Systems: Unused, Always 0
• 0: No action
• 1: Force bit 4 of status register to 1 "Not locked". May be used for keyboard testing after power on
• Bit 4: Keyboard Enable
• 0: Enable
• 1: Disables Keyboard by driving the clock line low
• Bit 5: Mouse Enable
• EISA and PS/2 Systems:
• 0: Enable mouse
• 1: Disable mouse by driving the clock line low
• ISA Systems:
• 0: Use 11 bit codes, check parity, and do scan convertions
• 1: Use 8086 codes, dont check parity nor do convertions
• Bit 6: Translate
• 0: No translation
• 1: Translate keyboard scancodes. MCA Type 2 controllers cannot set this bit to 1, so they have to set the conversition using command 0xF0 to Port 0x60.
• Bit 7: Not used

As you can imagine, this is very low level details. Not only are ports represented by numbers (That can represent virtually anything), but commands are too. Programmers of high level languages, such as C, know of the common off by one error. This error is very easy to make here. However, we are in Ring 0, so the results can be worse then applications.

Being off by one can either a) Make the controller do something you don't want, b) If the command is unkown: 1) Controller ignores it, or 2) Controller malfunctions; c) Controller just malfunctions, d) Physical hardware damage, or e) Triple fault.

This is important, so lets look deeper.

Cautions to look for

Controller executes the wrong command

Unkown Controller Command

Some controllers may malfunction, however. Please see the "Malfunction" section for more information.

Controller Malfunctions

The foof problem is more series. When the processor is given the command bytes 0xf0 0x0f 0xc7 0xc8, which is an example of an Hault and Catch Fire (HCF) instruction. (An Undocumented Instruction). Most of these instructions may lock up the processor itself, forcing the user to hard reboot. Others may cause unusual side effects from the use of these instructions.

One should coinsider these problems that may happen. It does happen, and controllers are no exception. (Remember that we send instruction bytes to individual ports? For example, Port 0x64--The Command Register in the Keyboard Controller).

Most of these malfunctions can easily be coinsidered "Design Flaws" of the device, though.

Physical Hardware damage

Triple Fault

Controller problems in Bochs

For example, if you try to send an unkown command (Such as 0) to the keyboard controller:

  
	mov		al, 0x00		; some random command
	out		0x64, al		; try to send command to controller
	

  
	[KBD   ] unsupported io write to keyboard port 64, value = 0
	

Device Drivers

Everything here is specific to the 8042 microcontroller, which is sometimes embedded onto the motherboard. Because this microcontroller is (Kind of) standard to PCs and laptops, our code will work on most systems, but there is no guarantee it will work on all of them. In fact, it won't work on all of them.

So--What will happen on those "other" computers? See the Cautions to look for section.

As you can see, there are a few big problems: a) This is very nonportable. And, b) Magic Numbers everywhere. The code will become very hard to maintain because of this. The numbers could represent port addresses, data, commands, registers, etc.. there is almost no way you can know without a comment of sorts.

Also, notice we have two registers, and a command byte. Each of the bits represent different things. Knowing this, and all of the above, you can imagine that this could get extremily complex. This is just a keyboard controller. And yet, we have not described everything in it yet!

Imagine how much more complex a video controller, or an IDE controller is. Wait... IDE Controller? Yep. You have to go through the IDE Controller to turn the hard drives on, And determin how much hard drives there are. And, no: Code for SATA Controllers are different then code for IDE Controllers, even if they both interact with hard drives.

This type of detail is simply never required in applications. However, for us, we need to work at this level. Yes, Indeed: It can get extremily complex.

To create an abstraction interface between the software and hardware will fix most of these problems, and make interaction to hardware far more easier.

A Device Driver is a Ring 0 program, executed by the OS, that does just this. The managment of loading drivers and a friendly envirement (To insure they do not crash) is handled by the OS.

In the case for Windows XP, Windows loads its Device Drivers at the start of the Kernel.

Hardware Abstraction Layers

Resetting the Computer - The right way

One way of resetting the computer is through the keyboard controller. Look back up at the Commands table, shown above. You should see 0xFE - System Reset. So: By sending command 0xFE to the command register (Port 0x64), you will reboot the computer. Heres an example:

  

	; Reset computer without triple fault

	mov	al, 0xFE		; reset command
	out	0x64, al		; send command to keyboard controller
	

We will look at reading and writing the keyboard buffer later when we develop a driver for it.

Enabling A20

Lets hide the ugly A20 stuff into its own file:

A20.inc

  



;********************************************
;	Enable A20 address line
;	ret/ EAX = 0 for success, -1 for failure
;
;	OS Development Series
;********************************************

bits	32							; pmode 32 bit code

global _EnableA20

_EnableA20:

	pusha							; save registers
	cli							; clear interrupts (dangerous code ahead)
	mov		cx, 5					; reset counter

; Here, we are looping 5 times. We set the counter (CX to 5). We are going to try
; 5 times to enable A20.

.EnableA20start:						; The beginning of the loop

;-------------------------------;
; Wait for controller		;
;-------------------------------;

; We looked at this code before. It simply loops until the status buffer has data in it to read from

.KKbrdCntrlWait:
	xor		ax, ax					; clear ax
	in		al, 64h					; read from status register--if bit 0 set, buffer empty
	bt		ax, 1					; is it set?
	jc		.KKbrdCntrlWait				; nope--wait some more.

;-------------------------------;
; Attempt to enable A20		;
;-------------------------------;

; We seen this code to! Here we send the command to the keyboard controller

	mov		al, 0dfh				; command 0xDF: Enable A20
	out		64h, al					; send command to keyboard controller

; Attempt to read back the status byte -------------

;-------------------------------;
; Wait for controller		;
;-------------------------------;

; Now, we have to wait again until data is in the buffer. This helps insure the new status of the controller,
; and the controller has completed our command to enable A20.

.KKbrdCntrlWait2:						; Wait some more. Notice this is the same as the above
	xor		ax, ax					; waiting game. Yes--The Keyboard Controller is slow.
	in		al, 64h
	bt		ax, 1
	jc		.KKbrdCntrlWait2

;-------------------------------;
; Read status byte		;
;-------------------------------;

; Remember that port 0x60 is the status register. Here we read in the status register from the controller

.KKbrdCntrlGetStat2:
	xor		ax, ax					; clear ax
	in		al, 60h					; read in status byte

;-------------------------------;
; Check if A20 is enabled	;
;-------------------------------;

; Now, just check bit 1 of the status register (Read into AX) to see if it was a success.

	bt		ax, 1					; check if bit 1 is set
	jc		.EnableA20success			; all good!
	
	loop		.EnableA20start				; No? Try again until CX=0. Decrements CX.

	jmp		.EnableA20fail				; If we get here, bail out

;-------------------------------;
; Success return		;
;-------------------------------;

.EnableA20success:
	popa							; restore registers
	xor		eax, eax				; return 0 -- success! :)
	ret

;-------------------------------;
; Error return			;
;-------------------------------;

.EnableA20fail:
	popa							; restore registers
	mov		eax, -1					; return -1 -- 5 attempts failed :(
	ret


	

Demo

  

;*******************************************************
;
;	Stage2.asm
;		Stage2 Bootloader
;
;	OS Development Series
;*******************************************************

bits	16

; Remember the memory map-- 0x500 through 0x7bff is unused above the BIOS data area.
; We are loaded at 0x500 (0x50:0)

org 0x500

jmp	main				; go to start

;*******************************************************
;	Preprocessor directives
;*******************************************************

%include "stdio.inc"			; basic i/o routines
%include "Gdt.inc"			; Gdt routines

;*******************************************************
;	Data Section
;*******************************************************

LoadingMsg db "Preparing to load operating system...", 0x0D, 0x0A, 0x00
msgCRLF     db 0x0D, 0x0A, 0x00
msgProgress db " ", 0x00
msgFailure  db 0x0D, 0x0A, "MISSING OPERATING SYSTEM. Press Any Key to Reboot", 0x0D, 0x0A, 0x00

datasector  dw 0x0000
cluster     dw 0x0000

;*******************************************************
;	STAGE 2 ENTRY POINT
;
;		-Store BIOS information
;		-Load Kernel
;		-Install GDT; go into protected mode (pmode)
;		-Jump to Stage 3
;*******************************************************

main:

	;-------------------------------;
	;   Setup segments and stack	;
	;-------------------------------;

	cli				; clear interrupts
	xor	ax, ax			; null segments
	mov	ds, ax
	mov	es, ax
	mov	ax, 0x9000		; stack begins at 0x9000-0xffff
	mov	ss, ax
	mov	sp, 0xFFFF
	sti				; enable interrupts

	;-------------------------------;
	;   Print loading message	;
	;-------------------------------;

	mov	si, LoadingMsg
	call	Puts16

	;-------------------------------;
	;   Install our GDT		;
	;-------------------------------;

	call	InstallGDT		; install our GDT

	;-------------------------------;
	;   Go into pmode		;
	;-------------------------------;

	cli				; clear interrupts
	mov	eax, cr0		; set bit 0 in cr0--enter pmode
	or	eax, 1
	mov	cr0, eax

	jmp	08h:Stage3		; far jump to fix CS. Remember that the code selector is 0x8!

	; Note: Do NOT re-enable interrupts! Doing so will triple fault!
	; We will fix this in Stage 3.

;******************************************************
;	ENTRY POINT FOR STAGE 3
;******************************************************

bits 32

; 32 bit includes ----------------------------------

%include "A20.inc"

Stage3:

	;-------------------------------;
	;   Set registers		;
	;-------------------------------;

	mov		ax, 0x10		; set data segments to data selector (0x10)
	mov		ds, ax
	mov		ss, ax
	mov		es, ax
	mov		esp, 90000h		; stack begins from 90000h

	;-------------------------------;
	;   Enable A20			;
	;-------------------------------;

	call		_EnableA20
	cmp		eax, -1
	je		Error


	jmp		STOP

;*******************************************************
;	Errors
;*******************************************************

Error:

	; print error message and halt.
	; Note: We cannot use our Puts16 routine here! Puts16, not only uses a different
	; addressing mode (seg:offset) then protected mode, but uses the BIOS to print
	; characters--both are bad, and will triple fault the cpu.

	; We will write our own versions of these routines in the next tutorial, when we talk
	; about directly writing to video memory.

	; For now, just fall into STOP and halt the system.


;*******************************************************
;	Stop execution
;*******************************************************

STOP:

	cli
	hlt
	

Download the latest demo (*.ZIP: 8KB) Here.

Conclusion

We looked at alot of new concepts here. We also got experience with hardware programming. Remember: This is the only way of communicating with hardware in protected mode! Good bye interrupts. Good bye BIOS. Good bye everything--we are now completely on our own.

Right now, you can probably start appreciating Windows a little more :) After all, they all had to start at our level.

Don't worry if you do not understand everything yet--It is complicated, I know. When we get to our Kernel, we are going to have an entire tutorial dedicated to programming the keyboard microcontroller, and writing a driver for it. Cool?

The next tutorial will be much easier. We are going to put Protected Mode on hold for now, and go back to the real mode code. We will add the FAT12 loading code to load our kernel. Now that A20 is enabled, we can load it at 1MB!

Also, we will get some BIOS information, and anything else that comes to mind :) See you there.

Until next time,

~Mike "Crypter"
BrokenThorn Entertainment. Currently developing EvolutionEngine and MicroOS Operating System.

Questions or comments? Feel free to Contact me.