BrokenThorn Entertainment

I've been reading the kernel/C++ tutorials, and noticed a bit of a blank. What exactly is the plan for deployment of the kernel, HAL, runtime, etc? You go over a lot of deep design theory, and then there's just some demos to download at the end which have already implemented things ahead of what is discussed. I also didn't see any sort of HAL in the one's I downloaded.

I'm not sure how the demos are MEANT to be deployed. Are they supposed to be built as a PE (*.exe) or just a flat lib? I also don't see how it's making any use of the small runtime set, and I've peeled it apart. The decorations on some of the kernel functions suggest they're meant to be called from the outside? Is that right? I've implemented my kernel as a multi-boot compliant PE executable, and I'm wondering how some of the calling conventions and such are going to work and how I need to structure my deployments. I'm just left with some unanswered questions from those tutorials! Mind you, I'm deviating from the tutorials a LOT, but it's a great help to understand what you are doing and how it all is meant to work before I go bushwacking into the unknown void*, lol. Is it really feasible/reasonable to have external code call functions that live inside a PE kernel? Or would that be better implemented with a "ask and receive" communicative type of design? (I think the latter.)

EDIT: Oops. Now I see the HAL code I somehow missed. Funny that it is totally unused in subsequent tutorials though. Guess that's what threw me off.

Hello,

atc wrote:and then there's just some demos to download at the end which have already implemented things ahead of what is discussed. I also didn't see any sort of HAL in the one's I downloaded.

None of the demos should have anything implemented from later chapters. The demos build off of the code from the previous chapters' demo and only adds the code discussed in that chapter.

The HAL in the series implements a basic motherboard driver implementing interfaces for the PIT, PIC, processor tables, DMA, and can be expanded to include APIC, RTC, CMOS, etc. The HAL is a static library and should be in all of the Kernel chapters starting from Chapter 15.

I'm not sure how the demos are MEANT to be deployed. Are they supposed to be built as a PE (*.exe) or just a flat lib?

Looking at the demo code, the kernel is an EXE. All other drivers are static libraries compiled into the kernel.

The decorations on some of the kernel functions suggest they're meant to be called from the outside? Is that right?

I apologize, I do not understand what you mean here.

I've implemented my kernel as a multi-boot compliant PE executable, and I'm wondering how some of the calling conventions and such are going to work and how I need to structure my deployments.

If this is for a real OS, I highly recommend against basing it off of any tutorial. Tutorials make things easier at the cost of not supporting more advanced features. This is true for any tutorial.

The answer to the above questions depends entirely on your OS design and structure. The series uses the following structure:
I can also Post Neptune's, our in house OS's deployment structure if you wish for an example of a "real OS" deployment structure.

but it's a great help to understand what you are doing and how it all is meant to work before I go bushwacking into the unknown void*, lol.

Everything is described in the tutorial text of how everything works. As for the direction of the series OS and where it is headed, I personally do not know myself. That is one of the fun thing about writing the series

Is it really feasible/reasonable to have external code call functions that live inside a PE kernel?

I personally do not recommend other code from external programs calling your kernel, however you can support DLL linkage for your kernel fairly easily.

Funny that it is totally unused in subsequent tutorials though. Guess that's what threw me off.

Its used in all later tutorials for system initialization. Without it, the later chapters would not be possible.

I hope I got everything

Yes, you got all of it I think. Half of my confusion was from mixing up which demos went with which tutorials. The other half is probably my bad insomnia, lol. Staying up for 24-48hrs has become all too common for me, lol...

But yes, I see now the tutorial kernel is a PE. The Neptune loader is even designed to accommodate that I see. I too figured it may be a bit dangerous to have a kernel which exposes its guts to the outside world; even if to only your privileged components. I'm going to work today on figuring out how to support DLLs properly. Basically, I want to have a compact interface to expose functionality with *very* loose coupling between components. Sort of the OS equivalent of a CAB (without clients of course). That will allow for easier repairs/upgrades/replacements/etc in the future.

I call my design philosophy "complex simplicity". Yes, an oxymoron, lol. But essentially it means that each individual part of the design is simple. But together, they form a complex application, framework or in this case, system which can handle complex tasks. They can be addressed on the simple/individual level without worrying about system level implications; so long as the basic contracts between them are not modified. For example, instead of just copy/pasting the demo code for writing text to the screen, I encapsulated everything into a "Debug" namespace and put all the I/O stuff in a "DebugIO" class and I'm totally re-writing it for extended functionality. I'm mainly a C# programmer, so I avoid "free functions" and objects unless there is a profound reason for them (i.e., some initialization which has no apparent 'home').

So anyway, I'm about to see what I can do to support DLLs and dynamic linking. Even static/implicit linking with DLLs is ok for ring 0 stuff (if nothing else needs to talk to it). Not really sure how to go about this just yet. I've been sheltered from such problems by APIs like Win32 and .NET my whole programming career. Yup, OS-n00b alert!

Hello,

The demo numbers do not match the chapter, I know It will be all updated in the 2010 revision to fix it so it does to reduce confusions.

Please note that Neptune is not at all related to the OS Development Series. Both boot programs can load PE executables, however are very different from each other. More information can be found on the Neptune web site.

Your design should work well; I personally believe that with simpler interfaces will make it easier to be able to extend and support more functionality with. It takes longer to work with but the end result is really nice.

DLLs are not that hard to support - feel free to post questions if anything comes to mind

Hello,

Sorry ATC, I edited your post by mistake and had to remove it.

Here is the response for your post:

Well, first questions I have. Since this isn't Windows, what do I need to do about a DLLMain(...) function? Is it necessary or advantageous?

Its up to you. The entry point for DLLs are optional - if you want to support and call it, it is the same as finding the entry point of EXEs. The name of the entry points can be anything you like as it is not a Windows compatible DLL that you are building.

I also saw (somewhere) where someone suggested you could implicitly/statically link to the DLL at build/compile time and then loading them into RAM manually wouldn't be necessary?

Incorrect. If they are delay loaded DLLs, Windows will automatically load the DLL for you. In the case of your own OS, you will need to load it still.

Or should functions just be called directly? How is that possible? Basically, I guess just a general rundown of how this should work would help.

If you just want to call a function from another DLL, you will need to basically write GetProcAddress() like method. This routine parses the DLL's export table for the function that you are wanting to find usually by name. When it finds it, it gets the functions RVA and adds it to the base of the DLL's (the location it was loaded to) and returns a void* to it. This can be converted to a function pointer by the EXE calling it.

Thus all the EXE has to do is something like void (*p) () = GetProcAddress (myDllBase, "myFunction"); p() to call the function myFunction() in the DLL.

This also means that any function that you would like exported from the DLL (usable by other programs) must have __declspec (dllexport).

This page includes nice information that can help you better understand what GetProcAddress has to do. More specifically, starting from "The Exports Section".

I can also post Neptune's GetProcAddress() for an example of how one might look like.

Yes, please! If it's excessively large, you can always attach it in .cpp/.c or whatever you've saved it as. I didn't hear that you don't need to load DLLs here. That was some other website where I read that, and I questioned the idea. Good to know before I wasted any time expecting it to work. I figured that was Windows specific, like the way .NET links with DLLs/.NET assemblies.

Paradox warning: This is funny though. The kernel will need to implement the functionality to load and call a DLL. THEN, it will need to be reimplemented again in a DLL! There's no way to just do it once in a DLL, because then the kernel can't call the DLL to use it! Freakin computers! Haha! Then that brings up another question. If it's in a DLL, how will a program call the DLL to use another DLL? LOL!

I've got to think about how I'll handle that now for my API. It's funny what comes to mind when you start to think outside of your every-day operating system and their comforts you take for granted.

Don't worry about editing the post. We all make mistakes!

EDIT: Oooh... Could there possibly be any legal consequences to using PEs and DLLs on a new operating system? I wouldn't think so, since the standards are out there and can be reimplemented (Mono, for one, has done it). Just got to ask. I'm going to look it up too and see if the formats have any legal restrictions.

Hello,

Paradox warning: This is funny though. The kernel will need to implement the functionality to load and call a DLL. THEN, it will need to be reimplemented again in a DLL!

What I do is impliment a separate library that is linked against the kernel that includes all functionality for working with EXEs. This gets rid of code duplication and works quite well.

If it's in a DLL, how will a program call the DLL to use another DLL? LOL!

The same way the kernel does it. Of course, the DLL the DLL relies on needs to be loaded to

A way to auto-load all DLLs needed is to, during loading, you can recursively parse each programs' import tables and obtain each DLL that the program relies on by name to load.

Oooh... Could there possibly be any legal consequences to using PEs and DLLs on a new operating system?

None that I know of. Knowing the amount of large scale systems that use them, I would not think so. Would certainly create a lot of legal issues with EFI and systems using it if it was.

That makes a bit more sense! Please post your example whenever you can. I'd love to look at it and get a better idea of what I'm up against. I did, however, make a rule for myself before starting this: NO copying outside code. Otherwise, I don't really learn jack. Inspiration is fine, and learning by example is fine. But I've banned Copy+Paste from my project.