Page 1 of 1

pmmngr memory allocation problem?

Posted: Tue Jul 10, 2012 3:16 pm
by xixpsychoxix
I have a weird error here. I am attempting to use the pmmngr/vmmngr to make a heap allocator. I used the code from K&R section 8.7 "Example: A Storage Allocator". The storage allocator in the book uses sbrk to get memory from the kernel so instead I rewrote it to use pmmngr_get_block and vmmngr_map_page to map the pages to virtual memory. My problem is whenever my allocator needs to get additional blocks from the physical memory manager I get a floating point exception? Here is the code for the memory manager:

Code: Select all

/* heap.cpp: Allocation code for my operating system. Uses a linked list to maintain heap data. */

// standard includes
#include <stdint.h>
#include <_null.h>

// kernel includes
#include <kernel/heap.h>
#include <kernel/mmngr_virt.h>
#include <kernel/mmngr_phys.h>

/* internal definitions/constants/macros */
#define BLOCK_SIZE 4096
typedef long Align;				// this helps us align memory headers on long boundaries

/* internal (private) structures/enums */
union heap_header {

	struct {
		heap_header  *next;			// point to the next allocation/free block
		unsigned int alloc_sz;		// allocation size
	} s;

	Align x;
};

/* internal (private) globals */
static heap_header _heap_base;
static heap_header *_heap_free=0;
static uint32_t _virt_addr = 0xb0000000;

/* internal (private) routines */
heap_header *morecore () {

	char *cp;
	heap_header *up;

	cp = (char *) pmmngr_alloc_block ();		// allocate more memory for the heap

	// if cp is null we have no memory left
	if (cp == NULL)
		return NULL;

	vmmngr_map_page (cp, (void *) _virt_addr);		// and map it's virtual address to it's physical address
	_virt_addr += BLOCK_SIZE;		// tack on nu bytes to the virtual address; this will be our next allocation address

	up = (heap_header *) cp;
	up->s.alloc_sz = BLOCK_SIZE;
	heap_free ((void *)(up+1));

	return _heap_free;
}

/* external interface (public) routine implementation */

// void *malloc (unsigned int nbytes): Allocate nbytes from the heap
// Inputs: nbytes-number of bytes to allocate
// Returns: pointer to a block of memory of nbytes; NULL on error
void *heap_alloc (unsigned int nbytes) {

	heap_header *p, *prev_p;	// used to keep track of the current unit
	unsigned int nunits;		// this is the number of "allocation units" needed by nbytes of memory

	nunits = (nbytes + sizeof (heap_header) - 1) / sizeof (heap_header) + 1;		// see how much we will need to allocate for this call

	// check to see if the list has been created yet; start it if not
	if ((prev_p = _heap_free) == NULL) {

		_heap_base.s.next     = _heap_free = prev_p = &_heap_base;		// point at the base of the memory
		_heap_base.s.alloc_sz = 0;										// and set it's allocation size to zero
	}

	// now enter a for loop to find a block fo memory
	for (p = prev_p->s.next;; prev_p = p, p = p->s.next) {

		// did we find a big enough block?
		if (p->s.alloc_sz >= nunits) {

			// the block is exact length
			if (p->s.alloc_sz == nunits)
				prev_p->s.next = p->s.next;

			// the block needs to be cut
			else {

				p->s.alloc_sz -= nunits;
				p += p->s.alloc_sz;
				p->s.alloc_sz = nunits;
			}

			_heap_free = prev_p;
			return (void *)(p+1);
		}

		// not enough space!! Try to get more from the kernel
		if (p == _heap_free) {

			// if the kernel has no more memory, return error!
			if ((p = morecore ()) == NULL)
				return NULL;
		}
	}
}

// void heap_free (void *ap): Free the block of memory at *ap from the heap
// Inputs: ap-memory block to free
// Returns: None
void heap_free (void *ap) {

	heap_header *bp, *p;
	bp = (heap_header *) ap-1;

	for (p = _heap_free; !(bp > p && bp < p->s.next); p = p->s.next) {

		if (p >= p->s.next && (bp > p || bp < p->s.next))
			break;
	}

	// join to upper allocation
	if (bp + bp->s.alloc_sz == p->s.next) {

		bp->s.alloc_sz += p->s.next->s.alloc_sz;
		bp->s.next = p->s.next->s.next;
	}

	else
		bp->s.next = p->s.next;

	if (p + p->s.alloc_sz == bp) {

		p->s.alloc_sz += bp->s.alloc_sz;
		p->s.next = bp->s.next;
	}

	else
		p->s.next = bp;

	_heap_free = p;
}
It is literally a copy of the one in K&R with about six or seven little modifications. Here is the code that calls it:

Code: Select all

int load_executable (char *cmd, int argCount, char *args) {

	// print a message regarding load attempt
	printf ("\nAttempting to load executable %s...\n", cmd);

	// map a 512-byte section at 0x1000
	unsigned char *executable = (unsigned char *) heap_alloc (512);
	vmmngr_map_page ((void *) executable, (void *) 0x1000);

	// read the file in
	FILE exe_file_ptr = volOpenFile (cmd);
	volReadFile (&exe_file_ptr, executable, 512);
	volCloseFile (&exe_file_ptr);

	// now cast the buffer and jump there
	((flat_exe)executable)();

	return 0;
}
I know this is overly simple code for loading executables but I am loading exes made in flat binary format from NASM. Here is the exe I am trying to load:

Code: Select all

org 0x1000
bits 32

        mov eax,0x12345678
        ret

assembled with:

nasm test.asm -o b:\test.exe -f bin

It seems to cause this problem whenever I try to allocate multiple blocks. I have a little more info to gather and I will post any updates I have. Advice is, as always, very appreciated!

Re: pmmngr memory allocation problem?

Posted: Tue Jul 10, 2012 3:24 pm
by xixpsychoxix
You know what, it seems to be working now! Though if anyone has advice as to what may have gone wrong before I would love to hear it! Thanks :D

Edit: AAH!!! I still have the issue. It seems I solved it here in my main routine (which I don't think it should have!!!):

Code: Select all

int _cdecl main (multiboot_info *boot_inf) {

	// get the size of the OS image first!

	_asm mov word ptr [kernelSize],dx

	// clear the display, print a test message, and return
	clearDisplay ();

	if (InitAbstractionLayer () != 0) {

		printf ("Sorry, there was an error initializing the HAL!\r");
		printf ("Shutting down...\r");

		return -1;
	}

	InitializeSystem (boot_inf);			// set up our HAL and interrupts, keyboard minidriver, floppy minidriver, and memory manager
	StartCommandCallbackSystem (def_cmd);	// start up our command line interface

	// register callback handlers for the command line
	RegisterCommandCallback ("help", (CommandCallback) help);
	RegisterCommandCallback ("cls",  (CommandCallback) cls);
	RegisterCommandCallback ("exit", (CommandCallback) exit);

	// register callback for executable files
	RegisterExeHandler ((ExeCallback) load_executable);

       /*** whenever these lines are taken out I get the FPU fault in the load_executable code!!! ***/
	char *d = (char *) pmmngr_alloc_blocks (10);
	printf ("I allocated d using the pmmngr: %d\n", d);
      
	char *e = (char *) heap_alloc (2048);
	printf ("e was allocated using heap_alloc (2048) and it is at %d\n", e);
       /*** from the last comment down to here! ***/

	run_kernel ();						// and run the command line interface for the kernel
	ShutdownAbstractionLayer ();		// shut down the system

	return 0;
}
Most specifically when I take out the code:

Code: Select all

char *d = (char *) pmmngr_alloc_blocks (10);
I get the floating point error. Is this because my pmmngr is not set up correctly? Any other necessary code, just ask!

Re: pmmngr memory allocation problem?

Posted: Wed Jul 11, 2012 12:52 am
by Mike
Hello,

It does indeed sound like the PMM is not initialized properly. If you are using either GrUB or the Neptune boot loader, kernelSize would contain garbage. Also, your boot_inf might be invalid as well unless you have another function that passes it into main.

It is better to reserve some area in the address space for the bitmap rather then just placing it at the end of the kernel in memory anyways. ie; you could reserve 0xe0000000-0xf0000000 (virtual address) for memory management structures. If you still want to get the kernelSize to place the PMM bitmap at the end, though, you can by getting the image size from the PE headers in memory. I do not see where the PMM is initialized so have to assume its a possible problem area.

Regarding brk() and sbrk(), you should implement similar functions in your heap allocator. brk() and sbrk() would reserve the memory for the heap so can be the only functions in the heap allocator to interact with the PMM directly. In other words, brk() would call the PMM and VMM to allocate and extend the heap. malloc() calls brk() when needed to allocate or expand the kernel heap.

Re: pmmngr memory allocation problem?

Posted: Wed Jul 11, 2012 11:57 am
by xixpsychoxix
Thanks Mike. I did switch back to using the tutorial loader since it works very well with the system as-is. Here is where the pmmngr gets initialized:

Code: Select all

void InitializeSystem (multiboot_info *boot_inf) {

	// if we got here we successfully initialized the abstraction layer. We can now initialize interrupt handlers
	setvect (0, (void(_cdecl &)(void)) divide_by_zero_exception);
	setvect (1, (void(_cdecl &)(void)) single_step_trap);
	setvect (2, (void(_cdecl &)(void)) nmi_trap);
	setvect (3, (void(_cdecl &)(void)) breakpoint_trap);
	setvect (4, (void(_cdecl &)(void)) overflow_trap);
	setvect (5, (void(_cdecl &)(void)) bounds_check_fault);
	setvect (6, (void(_cdecl &)(void)) invalid_opcode_fault);
	setvect (7, (void(_cdecl &)(void)) no_device_fault);
	setvect (8, (void(_cdecl &)(void)) double_fault_abort);
	setvect (9, (void(_cdecl &)(void)) invalid_tss_fault);
	setvect (10, (void(_cdecl &)(void)) no_segment_fault);
	setvect (11, (void(_cdecl &)(void)) stack_fault);
	setvect (12, (void(_cdecl &)(void)) general_protection_fault);
	setvect (13, (void(_cdecl &)(void)) page_fault);
	setvect (14, (void(_cdecl &)(void)) fpu_fault);
	setvect (15, (void(_cdecl &)(void)) alignment_check_fault);
	setvect (16, (void(_cdecl &)(void)) machine_check_abort);
	setvect (17, (void(_cdecl &)(void)) simd_fpu_fault);

	// get the size of memory
	uint32_t memSz = 1024+(boot_inf->memoryLow + boot_inf->memoryHi*64);
	pmmngr_init (memSz, 0x100000+(kernelSize*512));

	// display the memory map
	memory_region *region = (memory_region*) 0x1000;

	for (int i = 0; i < 15; i++) {

		// for any map types > 4 we mark them as reserved
		if (region[i].type > 4)
			region[i].type = 2;

		// if the starting address of the block is zero we are done!
		if (i > 0 && region[i].startLo == 0)
			break;

		// if the region is available, add it to the physical memory manager
		if (region[i].type == 1)
			pmmngr_init_region (region[i].startLo, region[i].sizeLo);
		else
			pmmngr_deinit_region (region[i].startLo, region[i].sizeLo);
	}

	// deinitialize kernel memory
	pmmngr_deinit_region (0x100000, kernelSize*512);
	vmmngr_initialize ();
	
	// enable interrupts
	enable ();

	kkybrd_install (33);			// install the keyboard to interrupt 33

	// setup the floppy disk controller
	flpydsk_set_working_drive (0);

	// we have to allocate a page to use for the DMA controller and floppy
	void *flpydsk_dma = pmmngr_alloc_block ();
	vmmngr_map_page (flpydsk_dma, flpydsk_dma);
	
	flpydsk_install (38);
	flpydsk_set_dma (flpydsk_dma);

	fsysFatInitialize();
	clearDisplay ();
}
I will adjust my pmm to use a different memory address for the memory bitmap. Also, shouldn't I be deinitializing the bitmap memory somewhere with the pmm? I can't remember if the pmmngr_init routine does that part or not.

Re: pmmngr memory allocation problem?

Posted: Fri Jul 13, 2012 12:16 am
by Mike
Hello,

pmmngr_init already reserves the space for the bitmap itself so you do not need to do it. Looking at the code, it looks to initialize the bitmap at 1mb+kernelSize*512 which is correct as the kernel binary is loaded in 512 byte segments. Does the code work "as-is" with the series boot loader?

Also, please note that the calculation for the size of memory might be unreliable. The best way of determining size would be from the memory map given by the BIOS.

Re: pmmngr memory allocation problem?

Posted: Fri Jul 13, 2012 1:30 am
by xixpsychoxix
The code as I have provided will work with the series bootloader, no problem. But again, as soon as I take that first pmmngr_alloc_block out of the main method I get an error. I don't understand because there are other places where the pmmngr is being called BEFORE this so I don't think it is an allocation issue, unless my memory map is wrong somewhere / memory size is wrong? But either way if the amount of memory was wrong I would be getting a page fault. I am getting an FPU fault...

Edit: I am now getting an invalid opcode fault? I don't know sometimes I get the FPU fault, sometimes the invalid opcode fault. Not sure what the problem is at this point.

Re: pmmngr memory allocation problem?

Posted: Fri Jul 13, 2012 3:43 am
by Mike
Hello,

The series identity maps the first 4MB of the address space. Your code does not reserve 0x1000 physical in the PMM bitmap so it gets allocated as free memory. load_executable trashes the page at 0x1000 corrupting the memory block previously allocated. Also, load_executable should not be using the kernel heap to allocate for the process. It should allocate free frames from the PMM, and then map those to 0x1000.

Possible ways to resolve this error is: (1) call the PMM twice during initialization. This allocates and reserves 0 and 0x1000 pages so they do not get allocated (this is what you are currently doing) ; (2) remove the identity mapping (recommended if you plan to go multitasking). The identity mapping is done by vmmngr_initialize.

I think this is the problem given the provided information. If you are certain this is not, however, please provide insight.