|Operating Systems Development Series|
This series is intended to demonstrate and teach operating system development from the ground up.
In the previous tutorial, we talked about basic VGA programming in protected mode, and even built a 1337 demo too!
This is the tutorial you have been waiting for. It builds directly on the all of the previus code, and loads our Kernel at te 1 MB mark, and executes our Kernel.
The Kernel is the most important part of our OS. The Kernel...We have talked a little about this mysterous foe before, havn't we? We will talk about the Kernel alot more in the next few tutorials, including design, structure, and development.
Right now, we already have everything set up... It's time to load the Kernel and say good bye to Stage 2!
Note: This tutorial requires a basic understanding of the Bootloaders 3 and 4 tutorials. We cover everything in detail here, but all of the concepts are explained in depth in the Bootloaders 3 and 4 Tutorials. If you have not read those tutorials, Please look at those tutorials first.
OS Development Series Tutorial 5: Bootloaders 3
OS Development Series Tutorial 6: Bootloaders 4
If you have read them, this tutorial should not be that hard.
A Basic Kernel StubThis is the Kernel we will load:
Okay, there is nothing much here. We will build on this program heavily in the next section.
Notice that it is all 32 bit. Sweet, huh? We are going to be out of the 16 bit world completely here.
For now, we just hault the system when we get to the Kernel.Please note that we will not be using this file probably at all in the rest of the series. Rather, we will be using a 32 bit C++ compilier. After we load the kernel image in memory, we can parse the file in memory for the kernel entry routine and call the C main() routine directly from our 2nd stage boot loader. Cool, huh? In other words, we will go from our 2nd stage boot loader directly into the C++ world without any stub file or program. However, we need a starting point. Because of this, we will use a basic stub file in this tutorial to help test and demenstrate it working. In the next few tutorials we will be getting our compiliers up and working and use that instead. But now we are getting ahead of ourselves here ;)
The floppy interfaceYey! Its time to finish off stage 2! In oder to load the Kernel we need to traverse FAT12 again. But before that, we have to get sectors off disk.
This code is EXACTALLY the same from our bootloader, and uses the BIOS INT 0x13 to load sectors off disk.
Because this tutorial is also a complete review, lets break each routine into sections and describe exactally what is going on.
Reading a sector - BIOS INT 0x13We talked about everything requarding loading sectors in our Bootloaders 3. Looking back at the tutorial, remember that we can use the BIOS Interrupt 0x13 function 2 to read a sector. Okay, then. The problem here is that We have to load sectors before going into protected mode. If we attempt to call a BIOS interrupt from protected mode, the processor will triple fault, remember?
Anyways, what was the interrupt? Right....
INT 0x13/AH=0x02 - DISK : READ SECTOR(S) INTO MEMORY
This is not THAT hard. Remember from the Bootloaders tutorial though. That is, we need to keep track of the sector, track, and head number, and insure we don't load attempt to load a sector beyond the track. That is, Remember that there are 18 sectors per track? Setting the sector number greater then 18 will cause the controller to fail, and processor to triple fault.
Okay...18 sectors per track. Remember that each sector if 512 bytes. Also, remember that there are 80 tracks per side.
Okay then! All of this information... Sectors per track, the number of tracks, number of heads, the size of a sector, completely depend on the disk itself. Remember that a sector does not NEED to be 512 bytes?
We describe everything in the OEM Parameter Block:
This should look familiar! Each member has been described in Tutorial 5--Please see that tutorial for a full detailed explination of everything here.
Now, all we need to have is a method so that we can load any number of sectors from disk to some location in memory. We immediately run into a problem though. Okay--We know what sector we want to load. However, BIOS INT 0x13 does not work with sectors. Okay, it does--but it also works with cylinders (Remember that a cylinder is just a head?) and tracks.
So what does this have to do with anything? Imagine if we want to load sector 20. We cannot directly use this number, because there are only 18 sectors per track. Attempting to read from the 20th sector on the current track will cause the floppy controller to fail, and processor to triple fault, as that sector does not exist. In order to read the 20'th sector, we have to read Track 2 Sector 2, Head 0 We will verify this later.
What this means as that, if we want to specify a sector to load, we need to convert our linear sector number into the exact cylinder, track, and sector location on disk.
Wait for it...Aha! Remember our CHS to LBA conversition routines?
Converting LBA to CHSThis should sound familiar, doesn't it? Linear Block Addressing (LBA) simply represents an indexed location on disk. The first block being 0, the second block being 1. In other words, LBA simply represents the sector number, beginning with 0, where each "block" is a single "sector".
Anywhoo...We have to find a way to convert this sector number (LBA) to the exact cylinder/head/secor location on disk. Remember this from Bootloaders 4 tutorial?
Some of our readers exclaimed this code was fairly tricky--and I am to admit it is. So, I am going to excplain it in detail here.
First, lets look at the forumlas again:
Okay! This is pretty easy, huh? The "logical sector" is the actual sector number we want. Note that the logical sector / sectors per track is inside of all of the above equations.
Because this division is inside of all of these equations, we can store it's result and use it for the other two expressions.
Lets put this into an example. We already said the 20th sector should be Track 2, Sector 2, remember? Lets try to put this formula to the test then:
We only keep the absolute number (2)--Aha! Sector 2! Note that we need to add 1 here because LBA addressing begins from 0. Remember that the basic formula "logical sector / sectors per track" is in ALL of these formulas. It is simply 1.1111111111111111111111111111111 in this example (Note in the above formula, we added 1 more). Because we are working with whole numbers, this is simply 1.
Remember from the OEM Block that we specified 2 heads per cylinder. So far, this indicates sector 2 on Head 1. Great--but what track are we on?
Notice that this is the exact same formula as above. The ONLY difference is that simple operation.
Anywhoo... following the formula we have: Logical Sector 20 is on Sector 2 Track 2 Head 0. Compare this with what we originally said in the previous section, and notice how this forumla works ;)
Okay, so now lets try to apply these formulas in the code:
LBACHS Explanation: Detail
Okay, this routine takes one parameter: AX, which contains the logical sector to convert into CHS. Note the formula (logical sector / sectors per track) is part of all three formulas. Rather then recalculating this over and over, it is more efficiant to just calculate it once, and use that result in all of the other calculations... This is how this routine works.
Now AX contains the logical sector / sectors per track operation.
Begin with sector 1 (Remember the + 1 in logical sector / sectors per track ?)
Clear DX. AX still contains the result of logical sector / sectors per track
Now for the formulas...
absolute head = (logical sector / sectors per track) MOD number of heads
absolute track = logical sector / (sectors per track * number of heads)
The multiplication results into a division by the number of heads. So the only difference between these two is the operation--one is division, and one is the remainder of that division (The Modulus).
Okay, lessee...What instruction can we use that could return both the remainder (MOD) and division result? DIV!
Remember that (logical sector / sectors per track) is still in AX, so all we need to do is divide by number of heads per cylinder...
The equtions for absolute head and absolute track are very simular. The only actual difference is the operation. This simple DIV instruction sets both DX and AX. AX Now stores the DIVISION of HeadsPerCylinder; DX now contains the REMAINDER (Modolus) of the same operation)
I hope this clears things up a bit. If not, please let me know ;)
Converting CHS to LBAThis is alot more simpler:
Reading in sectorsOkay, so now we have everything to read in sectors. This code is also exactally the same from the bootloader.
Okay, here we attempt to read the sectors 5 times.
We store the registers on the stack. The starting sector is a linear sector number (Stored in AX). Because we are using BIOS INT 0x13, We need to convert this to CHS before reading from the disk. So, we use our LBA to CHS coversition routine. Now, absoluteTrack contains the track number, absoluteSector contains the sector within the track, and absoluteHead contains the head number. All of this was set by our LBA to CHA conversition routine, remember?
Now we set up to read a sector, and envoke the BIOS to read it. For simplicity, lets take another look at the BIOS INT 0x13 routine that we are executing:
INT 0x13/AH=0x02 - DISK : READ SECTOR(S) INTO MEMORY
Compare this with how we execute the code above--fairly simple, huh?
Remember that the buffer to write to is in ES:BX, which INT 0x13 refrences as the buffer. We passed ES:BX into this routine, so that is the location to load the sectors to.
The BIOS INT 0x13 function 2 sets the Carry Flag (CF) is there is an error. If there is an error, decrement the counter (Remember we set up the loop to try 5 times?), and then try again!
If all 5 attemps failed (CX=0, Zero flag set), then we fall down to the INT 0x18 instruction:
...Which reboots the computer.
If the Carry Flag was NOT set (CF=0), then the jnz instruction jumps here, as it indicates that there was no error. The sector was read successfully.
Now, just restore the registers, and go to the next sector. Not to hard :) Note that, because ES:BX contains the address to load the sectors to, we need to increment BX by the bytes per sector to go to the next sector.
AX contained the starting sector to read from, so we need to increment that too.
I guess thats all for now. Please refrence Bootloaders 4 for a full explaination of this routine.
Floppy16.incIn the example demo, all of the floppy access routines are inside of Floppy16.inc.
FAT12 InterfaceYey--We can load sectors. Woohoo... :( As you know, we cannot really do much with that. What we need to do next is create a basic definition of a "file" and what a "file" is. We do this by means of a Filesystem.
Filesystems can get quite complex. Please refrence Bootloaders 4 while I explain this code to fully understand how this code works.
ConstantsDuring parsing Fat12, we will be needing a location to load the root directory table and the FAT table. To make things somewhat easier, lets hide these locations behind constants:
We will be loading our root directory table to 0x2e00 and our FAT to 0x2c00. FAT_SEG and ROOT_SEG are used for loading into segment registers.
Traversing FAT12As you know, some OS code can simply get ugly. Filesystem code, in my opinion, is one of them. This is one of the reasons why I decided to go over this code in this review-like tutorial. The FAT12 code is basically the same as the bootloaders, but I decided to modify it to decrease dependencies with the main program. Because of this, I decided to describe it in detail here.
Please note, I will not be going over FAT12 in detail here. Please see the Bootloaders 4 tutorial for complete details.
Anywhoo, as you know, in order to traverse FAT12 the first thing we need to load is the Root Directory Table, so lets look at that first.
Loading the Root Directory Table
Remember that the Root Directory Table is located right after the FAT's and Reserved sectors?
In loading the root directory table, we need to find a location in memory that we do not currently need and copy it there. For now, I chose 0x7E00 (Real mode: 0x7E0:0). This is right above our bootloader, which is still in memory because we have never overwritten it.
There is an important concept here. Notice that we have to load everything at absolute memory locations. This is very bad, as we have to physically keep track of where things are located. This is where a Low level memory manager comes into play. More later...
We first store the current state of the registers. Not doing so will effect the rest of the program that uses it, which is very bad.
Now we get the size of the root directory table, so that we know the number of sectors to load.
Remember from Bootloaders 4: Each entry is 32 bytes in size. When we add a new file in a FAT12 formatted disk, Windows automatically appends to the root directory for us, and adds to the bpbRootEntries byte offset varable of the OEM Parameter Block
See...Windows is nice :)
So...lessee, knowing each entry is 32 bytes in size, multiplying 32 bytes by the number of root directories will tell us how many bytes there are in the Root Directory Table. Simple enough, but we need the number of sectors--so we need to divide this result by the number of sectors:
OKAY, so now AX=number of sectors the root directory takes. Now, we have to find the starting location.
Remember from Bootloaders 4: The Root Directory table is Right after both FAT's and reserved sectors on the disk. Please look at the above disk structure table to see where the root directory table is located.
So...All we need to do is get the amount of sectors for the FAT's, and add that to the reserved sectors to get the exact location on disk:
Now that we have the number of sectors to read in, and the exact starting sector, lets read it in!
Notice that we set the seg:offset location to read into ROOT_SEG:0.
Next up, loading the FAT!
Loading the FATOkay...Remember from Bootloaders 4, we talked about the disk structure of a FAT12 formatted disk. Going Back in Time(tm), lets take another look:
Remember that there are either one or two FATs? Also notice that they are right after the reserved sectors on disk. This should look familar!
First we need to know how many sectors to load. Look back at the disk structure again. We store the number of FATs (and the sectors per FAT) in the OEM Parameter Block. So to get the total sectors, just multiply them:
Now, we need to take the reserved sectors into coinsideration, as they are before the FAT...
Yippe! Now, CX contains the number of sectors to load, so call our routine to load the sectors!
Thats all there is to it ;)
Searching for a fileIn searching for a file, we need the filename to search with. Remember that DOS uses 11 byte file names followng the common 8.3 naming convention (8 byte file name, 3 character extension.) Because of the way the enteries in the Root directory is structured, This MUST be 11 bytes--no exceptions.
Remember the format of the Root Directory Table: The filename is stored within the first 11 bytes of an entry. Lets take another look at the format of each directory entry:
Once we find a match, We need to refrence byte 26 of the entry to get it's current cluster. All of this should sound familiar.
Now...On to the code!
We first store the current register states. We need to use SI, so we need to save the current filename somewhere...BX, perhaps?
Remember that we need to parse the Root Directory table to find the image name. To do this, we need to check the first 11 bytes of each entry in the directory table to see if we found a match. Sounds simple, huh?
To do this, we need to know how many entries there are...
Okay, so CX now contains the number of entries to look in. All we need to do now is loop and compare the 11 byte character filename. Because we are using string instructions, we want to first insure the direction flag is cleared, which is what cld does.
DI is set to the current offset into the directory table. This is the location of the table. i.e., ES:DI points to the starting location of the table, so lets parse it!
If the 11 bytes match, the file was found. Because DI contains the location of the entry within the table, we immediately jump to .Found.
If it does not match, we need to try the next entry in the table. We add 32 bytes onto DI. (Remember that each entry is 32 bytes?)
If the file was not found, restore only the registers that are still on the stack, and return -1 (error)
If the file was found, restore all of the registers. AX contains the entry location within the Root Directory Table so that it can be loaded.
Yey! Now that we can find the file (and get it's location within the Root Directory Table), lets load it!
Loading a fileNow that everything is finally set up, it is finally time to load the file!
Most of this is pretty easy, as it calls our other routines. It is here that we loop, and insure that all of the file's clusters are loaded into memory.
Here we just save the registers. We need to keep a copy of the buffer to write to somewhere, so we keep that on the stack as well. CX is used to keep track of how many sectors we have loaded. We store this on the stack for later.
In loading the file, we will need to first find it (Kind of obvious, don't you think? ^^) We can easily use our FindFile routine here. FindFile sets AX to -1 on error, or the starting entry location within the Root Directory Table upon success. We can use this index to get anything we ever wanted to know about the file.
Okay, so if we get here, the file was found. ES:DI contains the location of the first root entry, which was set by FindFile(), so by refrencing ES:DI we effectivly get the file's entry.
Look back at the entry description table above in the previous section. Notice that we can offset 0x1A bytes to get to byte 26 (The starting cluster number), so store it...
The above is messy, I know. Remember that AX was set to the entry number by the call to FindFile? We need to store that here, but need to keep the buffer to write to on the top of the stack still. This is why I played with the stack a little here :)
Anyways, next we load the FAT. This is incredably easy...
OKAY then! Now that the FAT is loaded, and that we have the starting file cluster, it is time to actually read in the file's sectors.
This code is not that bad. Remember that, for FAT12, each cluster is just 512 bytes? i.e., each cluster simply represents a "sector". We first get the starting cluster/sector number. We cannot do much with just a cluster number though, as it is a linear number. That is, it is the sector number in CHS Not LBA format--It assumes we have the track and head information. Because our ReadSectors() requires an LBA linear sector number, We convert this CHS to an LBA address. Then, get the sectors per cluster, and read it in!
Note that we pop ES and BX--They were pushed on the stack from the beginning. ES:BX points to the ES:BP buffer that was passed to this routine--It contains the buffer to load the sectors into.
OKAY, so now that a cluster was loaded, we have to check with the FAT to determin if the end of file is reached. However, Remember that each FAT entry is 12 bytes? We found out from Bootloaders 4 that there is a pattern when reading the FAT:
For every even cluster, take the low twelve bits; for every high cluster take the heigh twelve bits
Please see Bootloaders 4 to see this in detail.
To determin if it is even or odd, just divide by 2:
Thats all there is too it! Granted a little complex, but not to hard, I hope ;)
Fat12.incGreat! All of the FAT12 code is in Fat12.inc.
Finishing Stage 2
Back to Stage 2 - Loading and Executing the Kernel Now that the messy code is over, all we need to do is load our Kernel image into memory from Stage 2, and execute our kernel. The problem is: Where?While we do want to load it to 1MB, we cannot do this directly yet. The reason is that we are still in real mode. Because of this, we will first need to load the image to a lower address first. After we switch into protected mode, we can copy our kernel to a new location. This can be 1MB, or even 3GB if paging is enabled.
Now our kernel is loaded to IMAGE_RMODE_BASE:0. ImageSize containes the number of sectors loaded (The size of the kernel). To execute inside of protected mode, all we need to do is jump or call it. Because we want our kernel at 1MB, we first need to copy it before we execute it:
There is a little problem here, though. This assumes the Kernel is a pure binary file. We cannot have this, because C does not support this. We need the Kernel to be a binary format that C supports, and we will need to parse it in order to load the Kernel using C. For now, we will keep this pure binary, but will fix this within the next few tutorials. Sound cool?
Our pure uber-1337 32 bit Kernel executing.
W00t! It's Kernel time!! :)
This tutorial covered alot of new code. Most of the concepts in tis tutorial we have went over before, so I hope this tutorial was not that hard ;)
In this tutorial, we covered these concepts in a new perspective, however. This may help understanding these topics a little bit more, and to see them being implimented into seperate routines.
We have developed code to load sectors off disk, and parse FAT12 to load our Kernel at whatever location we want. Cool, huh? In this Series, we are loading the Kernel at 1 MB.
With a basic full 32 bit Kernel finally loaded and executing, we can finally start focusing our attention to the most important part of any operating system -- The Kernel.
In the next few tutorials, we will cover Kernel Theory, Revolutions, and Designs. We will then start covering Low Level C Programming, and Low level programming with high level language concepts and theory.
There are Alot of freedom when programming C at Kernel Level, that most other programming fields do not allow. For example, There still is no such thing as an "Access Violation", so you still have direct control over every byte in memory. The bad news: There is also no such thing as a "standard library" either. To add more bad news, you still have to remember that you are programming a low level envirement, just with another abstraction layer that is C.
We will cover everything within the next few tutorials, and setting up C to work with our Kernel. I cannot wait!
See you there.... ;)
Until next time,