Page 1 of 1

Tutorial 17: Paging and Virtual Memory

Posted: Sun May 25, 2008 10:19 pm
by Mike
Hey everyone,

Tutorial 17 is almost ready for release. It covers:

<ul><li>Physical Memory</li><li>Virtual Memory</li><li>Memory Management Unit (MMU)</li><li>PAE and PSE</li><li>Paging Methods</li><li>Pages and Page Faults</li><li>The Page Table</li><li>The Page Directory Table</li><li>Implementing Paging</li></ul>

It goes into alot of detail on both physical memory and virtual memory, virtual addressing, and paging.

It also covers different methods of paging, page table types, and more.

After this we can start getting proper memory management into the kernel, which is what we will look at in the next tutorial.

We have also updated Tutorial 7 to cover the control registers in more detail.

Posted: Mon May 26, 2008 1:36 pm
by pathos
w00t

Posted: Thu Jun 05, 2008 5:12 pm
by pathos
Any word on a release date?

Posted: Fri Jun 06, 2008 5:25 am
by juFo
nice! keep up the good work!

Posted: Mon Jun 09, 2008 6:50 pm
by Mike
pathos wrote:Any word on a release date?
Sorry, not yet. The tutorial itself is getting rewritten for more and better content and to make it easier to read and follow. Also, we decided to add paging support in both the bootloader and kernel.

I will have very little time to work on it the next two weeks, Sorry :(

Posted: Tue Jun 17, 2008 8:20 am
by juFo
no problem, you are doing just fine ^^

Don't have any time either this week :)

Posted: Mon Jun 23, 2008 2:33 am
by Mike
Hm... Perhaps I started this thread to soon :/

Oh well.. I am back and the tutorial text is practically completed. I am planning it for release this week for comments and suggestions on improving it.

The demo might be a little longer however. The bootloaders paging support is good. Im currently still deciding on the final interface for the kernel however. Because of this, the tutorial might be released a little sooner then the demo again.

I decided to have the bootloader execute the kernel at a 3GB virtual address similar to most higher end operating systems. It will also help demonstrate a way of doing it, and provide examples of implementing paging in assembly language as well as C for better understanding.

If anyone has any thoughts or suggestions or ideas for the upcoming tutorial or demo, feel free to let me know :D

Posted: Tue Jun 24, 2008 4:11 am
by Mike
Tutorial has been uploaded.

Please take note the demo might not be up for another week or so while I decide on the designs for the interfaces.

Please let me know if its to complex, or if there are anything you would change!

As always, any suggestions or ideas are greatly appreciated by me and I am certain our readers as well. :)

Posted: Wed Jun 25, 2008 11:04 pm
by Andyhhp
Wow thats alot to take in - I shall have to read it properly when I have more time.

Sorry I have been inactive lately but I had exams then started with with Symbian - I got the job btw (Thanks).

In the meantime I offer the following as a minor correction.

In the paragraph:
Physical Memory is an abstract block of memory stored within the computers Random Access Memory (RAM). The way physical memory is "stored" within the RAM depends on the type of RAM the system uses. For example, Dynamic Random Access Memory (DRAM) stores each bit of data in its own capacitor that must be refreshed periodically. A capacitor is an electronic device that stores a current for a limited time. This allows it to either store a current (a binary 1), or no current (a binary 0). This is how DRAM chips store individual bits of data in a computer.
you say that capacitors store current.

Capacitors store charge, not current. Charge is a physical unit that depends on the number of electrons in a defined area. Current is the rate of change of charge, or the speed of movement of charge (in the same way velocity is the rate of change of displacement). Quite literally, the current is the differential of charge with respect to time. This is where some of the physics equations come from such as:
Q = IT
Charge = Current * Time
and
I = d(Q)/dt
Current = differential of Charge with respect to Time

But I think I have over-exaggerated the point so lets move away from physics onto a query:P

In the tutorial, you said that each program can have its own virtual address space. However, when we covered paging and virtual memory in OS lectures, we were told that only a single page table directory exists for all of memory, and this never gets replaces when paging needs to happen.

Is it the case that each program (i.e. process or thread depending on a windows or linux view) has its own virtual address table, which would result in wasted 4mb of memory per process, or is there a different mechanism by which a process can have its base at 0virtual yet not clash with all the other processes out there?

Thank for the great work,

Andrew

Posted: Fri Jun 27, 2008 7:31 pm
by Mike
Wow thats alot to take in - I shall have to read it properly when I have more time.
Is that a good thing or bad..? Do you think it might still be too complex?
Sorry I have been inactive lately but I had exams then started with with Symbian - I got the job btw (Thanks).
No worries--Its great to hear :D
Capacitors store charge, not current.
I am going to have to admit that I always get current and charge mixed up. Thanks for the correction--I am going to fix it on the next revision (When the demo is uploaded)
In the tutorial, you said that each program can have its own virtual address space.
This is true...From that programs perspective, anyways.

The way it is normally done is indeed the way you described it. While the page tables are 4MB in size per process, keep in mind that you dont need to have them all in physical memory. ie, you can cache page tables to disk freeing up more physical memory for more active processes.

Posted: Sat Jun 28, 2008 3:55 am
by Mike
Looks like we made a slight error in the bootloaders loading code as well.

I will update all tutorials and demos with the new code.

The following code fixes a possible triple fault problem with different optimization levels that can modify the entry point location.

The following also allows us to NOT have to use Order.txt at all. ie, we can omit it entirely.

The following code also extracts the image base from the program file, so we dont need to hardcode it.

The original code did not properly extract the entry point address nor image base from the file. (Sorry, I was off by 4 bytes in the original code :oops: ) The new code does just this.

Code: Select all

mov	ebx, [0x100000+60]  ; e_lfanew is a 4 byte offset address of the PE header; it is 60th byte. Get it
	
add ebx, 0x100000         ; Add base address. EBX now points to file sig (PE00)

; Here is where the slight changes are at:

add	ebx, 24
mov	eax, [ebx]          ; _IMAGE_FILE_HEADER is 20 bytes + size of sig (4 bytes)
add	ebx, 20-4           ; address of entry point
mov	ebp, dword [ebx]    ; get entry point offset in code section	
add	ebx, 12	          ; image base is offset 12 bytes from entry point
mov	eax, dword [ebx]    ; add image base
add	ebp, eax

jmp	ebp                 ; Execute Kernel
I am going to update all the tutorials with the error fix.

Posted: Tue Jul 01, 2008 10:33 pm
by Andrew
bah ima have to start with tutorial 1 =] i need to get back up to date lol how you doin mike? hey that message u emailed me i cannot find it, i saw yahoo post it on my email but i searched and its not there and i didnt delete try sending it again please.

Posted: Thu Jul 24, 2008 9:00 am
by Andyhhp
Hi,

Having finally got round to continuing my OS, I have a question.

Am I correct in assuming that when you are using Virtual Addressing, you completely ignore Descriptor:Offset addressing, and therefore dont need to set up a GDT if you go straight to Virtual Addressing in the 2nd stage bootloader?

Thanks,

Andrew

Posted: Fri Jul 25, 2008 4:02 am
by Mike
Virtual addresses are translated into physical addresses that follow the descriptor:offset addressing mode used by protected mode.

Because of this, a GDT is still required as pmode fundamentally requires it.