pmmngr memory allocation problem?

If you are new to OS Development, plan on spending some time here first before going into the other forums.

Moderator: Moderators

pmmngr memory allocation problem?

Postby xixpsychoxix » Tue Jul 10, 2012 3:16 pm

I have a weird error here. I am attempting to use the pmmngr/vmmngr to make a heap allocator. I used the code from K&R section 8.7 "Example: A Storage Allocator". The storage allocator in the book uses sbrk to get memory from the kernel so instead I rewrote it to use pmmngr_get_block and vmmngr_map_page to map the pages to virtual memory. My problem is whenever my allocator needs to get additional blocks from the physical memory manager I get a floating point exception? Here is the code for the memory manager:

Code: Select all
/* heap.cpp: Allocation code for my operating system. Uses a linked list to maintain heap data. */

// standard includes
#include <stdint.h>
#include <_null.h>

// kernel includes
#include <kernel/heap.h>
#include <kernel/mmngr_virt.h>
#include <kernel/mmngr_phys.h>

/* internal definitions/constants/macros */
#define BLOCK_SIZE 4096
typedef long Align;            // this helps us align memory headers on long boundaries

/* internal (private) structures/enums */
union heap_header {

   struct {
      heap_header  *next;         // point to the next allocation/free block
      unsigned int alloc_sz;      // allocation size
   } s;

   Align x;
};

/* internal (private) globals */
static heap_header _heap_base;
static heap_header *_heap_free=0;
static uint32_t _virt_addr = 0xb0000000;

/* internal (private) routines */
heap_header *morecore () {

   char *cp;
   heap_header *up;

   cp = (char *) pmmngr_alloc_block ();      // allocate more memory for the heap

   // if cp is null we have no memory left
   if (cp == NULL)
      return NULL;

   vmmngr_map_page (cp, (void *) _virt_addr);      // and map it's virtual address to it's physical address
   _virt_addr += BLOCK_SIZE;      // tack on nu bytes to the virtual address; this will be our next allocation address

   up = (heap_header *) cp;
   up->s.alloc_sz = BLOCK_SIZE;
   heap_free ((void *)(up+1));

   return _heap_free;
}

/* external interface (public) routine implementation */

// void *malloc (unsigned int nbytes): Allocate nbytes from the heap
// Inputs: nbytes-number of bytes to allocate
// Returns: pointer to a block of memory of nbytes; NULL on error
void *heap_alloc (unsigned int nbytes) {

   heap_header *p, *prev_p;   // used to keep track of the current unit
   unsigned int nunits;      // this is the number of "allocation units" needed by nbytes of memory

   nunits = (nbytes + sizeof (heap_header) - 1) / sizeof (heap_header) + 1;      // see how much we will need to allocate for this call

   // check to see if the list has been created yet; start it if not
   if ((prev_p = _heap_free) == NULL) {

      _heap_base.s.next     = _heap_free = prev_p = &_heap_base;      // point at the base of the memory
      _heap_base.s.alloc_sz = 0;                              // and set it's allocation size to zero
   }

   // now enter a for loop to find a block fo memory
   for (p = prev_p->s.next;; prev_p = p, p = p->s.next) {

      // did we find a big enough block?
      if (p->s.alloc_sz >= nunits) {

         // the block is exact length
         if (p->s.alloc_sz == nunits)
            prev_p->s.next = p->s.next;

         // the block needs to be cut
         else {

            p->s.alloc_sz -= nunits;
            p += p->s.alloc_sz;
            p->s.alloc_sz = nunits;
         }

         _heap_free = prev_p;
         return (void *)(p+1);
      }

      // not enough space!! Try to get more from the kernel
      if (p == _heap_free) {

         // if the kernel has no more memory, return error!
         if ((p = morecore ()) == NULL)
            return NULL;
      }
   }
}

// void heap_free (void *ap): Free the block of memory at *ap from the heap
// Inputs: ap-memory block to free
// Returns: None
void heap_free (void *ap) {

   heap_header *bp, *p;
   bp = (heap_header *) ap-1;

   for (p = _heap_free; !(bp > p && bp < p->s.next); p = p->s.next) {

      if (p >= p->s.next && (bp > p || bp < p->s.next))
         break;
   }

   // join to upper allocation
   if (bp + bp->s.alloc_sz == p->s.next) {

      bp->s.alloc_sz += p->s.next->s.alloc_sz;
      bp->s.next = p->s.next->s.next;
   }

   else
      bp->s.next = p->s.next;

   if (p + p->s.alloc_sz == bp) {

      p->s.alloc_sz += bp->s.alloc_sz;
      p->s.next = bp->s.next;
   }

   else
      p->s.next = bp;

   _heap_free = p;
}


It is literally a copy of the one in K&R with about six or seven little modifications. Here is the code that calls it:

Code: Select all
int load_executable (char *cmd, int argCount, char *args) {

   // print a message regarding load attempt
   printf ("\nAttempting to load executable %s...\n", cmd);

   // map a 512-byte section at 0x1000
   unsigned char *executable = (unsigned char *) heap_alloc (512);
   vmmngr_map_page ((void *) executable, (void *) 0x1000);

   // read the file in
   FILE exe_file_ptr = volOpenFile (cmd);
   volReadFile (&exe_file_ptr, executable, 512);
   volCloseFile (&exe_file_ptr);

   // now cast the buffer and jump there
   ((flat_exe)executable)();

   return 0;
}


I know this is overly simple code for loading executables but I am loading exes made in flat binary format from NASM. Here is the exe I am trying to load:

Code: Select all
org 0x1000
bits 32

        mov eax,0x12345678
        ret



assembled with:

nasm test.asm -o b:\test.exe -f bin

It seems to cause this problem whenever I try to allocate multiple blocks. I have a little more info to gather and I will post any updates I have. Advice is, as always, very appreciated!
xixpsychoxix
 
Posts: 59
Joined: Tue Oct 13, 2009 8:49 pm

Re: pmmngr memory allocation problem?

Postby xixpsychoxix » Tue Jul 10, 2012 3:24 pm

You know what, it seems to be working now! Though if anyone has advice as to what may have gone wrong before I would love to hear it! Thanks :D

Edit: AAH!!! I still have the issue. It seems I solved it here in my main routine (which I don't think it should have!!!):

Code: Select all
int _cdecl main (multiboot_info *boot_inf) {

   // get the size of the OS image first!

   _asm mov word ptr [kernelSize],dx

   // clear the display, print a test message, and return
   clearDisplay ();

   if (InitAbstractionLayer () != 0) {

      printf ("Sorry, there was an error initializing the HAL!\r");
      printf ("Shutting down...\r");

      return -1;
   }

   InitializeSystem (boot_inf);         // set up our HAL and interrupts, keyboard minidriver, floppy minidriver, and memory manager
   StartCommandCallbackSystem (def_cmd);   // start up our command line interface

   // register callback handlers for the command line
   RegisterCommandCallback ("help", (CommandCallback) help);
   RegisterCommandCallback ("cls",  (CommandCallback) cls);
   RegisterCommandCallback ("exit", (CommandCallback) exit);

   // register callback for executable files
   RegisterExeHandler ((ExeCallback) load_executable);

       /*** whenever these lines are taken out I get the FPU fault in the load_executable code!!! ***/
   char *d = (char *) pmmngr_alloc_blocks (10);
   printf ("I allocated d using the pmmngr: %d\n", d);
     
   char *e = (char *) heap_alloc (2048);
   printf ("e was allocated using heap_alloc (2048) and it is at %d\n", e);
       /*** from the last comment down to here! ***/

   run_kernel ();                  // and run the command line interface for the kernel
   ShutdownAbstractionLayer ();      // shut down the system

   return 0;
}


Most specifically when I take out the code:

Code: Select all
char *d = (char *) pmmngr_alloc_blocks (10);


I get the floating point error. Is this because my pmmngr is not set up correctly? Any other necessary code, just ask!
xixpsychoxix
 
Posts: 59
Joined: Tue Oct 13, 2009 8:49 pm

Re: pmmngr memory allocation problem?

Postby Mike » Wed Jul 11, 2012 12:52 am

Hello,

It does indeed sound like the PMM is not initialized properly. If you are using either GrUB or the Neptune boot loader, kernelSize would contain garbage. Also, your boot_inf might be invalid as well unless you have another function that passes it into main.

It is better to reserve some area in the address space for the bitmap rather then just placing it at the end of the kernel in memory anyways. ie; you could reserve 0xe0000000-0xf0000000 (virtual address) for memory management structures. If you still want to get the kernelSize to place the PMM bitmap at the end, though, you can by getting the image size from the PE headers in memory. I do not see where the PMM is initialized so have to assume its a possible problem area.

Regarding brk() and sbrk(), you should implement similar functions in your heap allocator. brk() and sbrk() would reserve the memory for the heap so can be the only functions in the heap allocator to interact with the PMM directly. In other words, brk() would call the PMM and VMM to allocate and extend the heap. malloc() calls brk() when needed to allocate or expand the kernel heap.
User avatar
Mike
Site Admin
 
Posts: 463
Joined: Sat Oct 20, 2007 7:58 pm

Re: pmmngr memory allocation problem?

Postby xixpsychoxix » Wed Jul 11, 2012 11:57 am

Thanks Mike. I did switch back to using the tutorial loader since it works very well with the system as-is. Here is where the pmmngr gets initialized:

Code: Select all
void InitializeSystem (multiboot_info *boot_inf) {

   // if we got here we successfully initialized the abstraction layer. We can now initialize interrupt handlers
   setvect (0, (void(_cdecl &)(void)) divide_by_zero_exception);
   setvect (1, (void(_cdecl &)(void)) single_step_trap);
   setvect (2, (void(_cdecl &)(void)) nmi_trap);
   setvect (3, (void(_cdecl &)(void)) breakpoint_trap);
   setvect (4, (void(_cdecl &)(void)) overflow_trap);
   setvect (5, (void(_cdecl &)(void)) bounds_check_fault);
   setvect (6, (void(_cdecl &)(void)) invalid_opcode_fault);
   setvect (7, (void(_cdecl &)(void)) no_device_fault);
   setvect (8, (void(_cdecl &)(void)) double_fault_abort);
   setvect (9, (void(_cdecl &)(void)) invalid_tss_fault);
   setvect (10, (void(_cdecl &)(void)) no_segment_fault);
   setvect (11, (void(_cdecl &)(void)) stack_fault);
   setvect (12, (void(_cdecl &)(void)) general_protection_fault);
   setvect (13, (void(_cdecl &)(void)) page_fault);
   setvect (14, (void(_cdecl &)(void)) fpu_fault);
   setvect (15, (void(_cdecl &)(void)) alignment_check_fault);
   setvect (16, (void(_cdecl &)(void)) machine_check_abort);
   setvect (17, (void(_cdecl &)(void)) simd_fpu_fault);

   // get the size of memory
   uint32_t memSz = 1024+(boot_inf->memoryLow + boot_inf->memoryHi*64);
   pmmngr_init (memSz, 0x100000+(kernelSize*512));

   // display the memory map
   memory_region *region = (memory_region*) 0x1000;

   for (int i = 0; i < 15; i++) {

      // for any map types > 4 we mark them as reserved
      if (region[i].type > 4)
         region[i].type = 2;

      // if the starting address of the block is zero we are done!
      if (i > 0 && region[i].startLo == 0)
         break;

      // if the region is available, add it to the physical memory manager
      if (region[i].type == 1)
         pmmngr_init_region (region[i].startLo, region[i].sizeLo);
      else
         pmmngr_deinit_region (region[i].startLo, region[i].sizeLo);
   }

   // deinitialize kernel memory
   pmmngr_deinit_region (0x100000, kernelSize*512);
   vmmngr_initialize ();
   
   // enable interrupts
   enable ();

   kkybrd_install (33);         // install the keyboard to interrupt 33

   // setup the floppy disk controller
   flpydsk_set_working_drive (0);

   // we have to allocate a page to use for the DMA controller and floppy
   void *flpydsk_dma = pmmngr_alloc_block ();
   vmmngr_map_page (flpydsk_dma, flpydsk_dma);
   
   flpydsk_install (38);
   flpydsk_set_dma (flpydsk_dma);

   fsysFatInitialize();
   clearDisplay ();
}


I will adjust my pmm to use a different memory address for the memory bitmap. Also, shouldn't I be deinitializing the bitmap memory somewhere with the pmm? I can't remember if the pmmngr_init routine does that part or not.
xixpsychoxix
 
Posts: 59
Joined: Tue Oct 13, 2009 8:49 pm

Re: pmmngr memory allocation problem?

Postby Mike » Fri Jul 13, 2012 12:16 am

Hello,

pmmngr_init already reserves the space for the bitmap itself so you do not need to do it. Looking at the code, it looks to initialize the bitmap at 1mb+kernelSize*512 which is correct as the kernel binary is loaded in 512 byte segments. Does the code work "as-is" with the series boot loader?

Also, please note that the calculation for the size of memory might be unreliable. The best way of determining size would be from the memory map given by the BIOS.
User avatar
Mike
Site Admin
 
Posts: 463
Joined: Sat Oct 20, 2007 7:58 pm

Re: pmmngr memory allocation problem?

Postby xixpsychoxix » Fri Jul 13, 2012 1:30 am

The code as I have provided will work with the series bootloader, no problem. But again, as soon as I take that first pmmngr_alloc_block out of the main method I get an error. I don't understand because there are other places where the pmmngr is being called BEFORE this so I don't think it is an allocation issue, unless my memory map is wrong somewhere / memory size is wrong? But either way if the amount of memory was wrong I would be getting a page fault. I am getting an FPU fault...

Edit: I am now getting an invalid opcode fault? I don't know sometimes I get the FPU fault, sometimes the invalid opcode fault. Not sure what the problem is at this point.
xixpsychoxix
 
Posts: 59
Joined: Tue Oct 13, 2009 8:49 pm

Re: pmmngr memory allocation problem?

Postby Mike » Fri Jul 13, 2012 3:43 am

Hello,

The series identity maps the first 4MB of the address space. Your code does not reserve 0x1000 physical in the PMM bitmap so it gets allocated as free memory. load_executable trashes the page at 0x1000 corrupting the memory block previously allocated. Also, load_executable should not be using the kernel heap to allocate for the process. It should allocate free frames from the PMM, and then map those to 0x1000.

Possible ways to resolve this error is: (1) call the PMM twice during initialization. This allocates and reserves 0 and 0x1000 pages so they do not get allocated (this is what you are currently doing) ; (2) remove the identity mapping (recommended if you plan to go multitasking). The identity mapping is done by vmmngr_initialize.

I think this is the problem given the provided information. If you are certain this is not, however, please provide insight.
User avatar
Mike
Site Admin
 
Posts: 463
Joined: Sat Oct 20, 2007 7:58 pm


Return to Beginning OS Development

Who is online

Users browsing this forum: No registered users and 1 guest

cron