I'm coding an operating system

Questions about programming languages and debugging
User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

I'm coding an operating system

Post by Gogeta70 »

So i've been developing an operating system lately. My intentions aren't to make a state-of-the-art, or even a very useful operating system. Rather, i'm simply doing this as a learning experience. I'll periodically update this post about my progress with it, and you can grab the latest source code and iso image to run the operating system yourself.

Image

Currently, the operating system is pretty simplistic. However, i have come a long way with it already... here's where i'm at:

-It runs in 32 bit protected mode
-I've enabled the capability to access 4Gb of RAM
-I've installed a Global Descriptor Table which allows me to access those 4Gb of RAM
-I've written a (very) basic text-mode VGA driver for displaying text on screen.
-I've implemented a few standard c library functions (strlen, strcat, itoa)
-Created an Interrupt Descriptor Table, so the CPU doesn't reset whenever it throws an exception
-Keyboard input
-Console
-Write a physical memory manager
-Write a Virtual memory manager
-Multi-threaded, context-switching capability

Here's what's in the works right now:

-Writing a BGA (Bochs Graphics Adapter) Driver. This is quicker, and works in both Bochs (duh) and (as far as i know) VirtualBox

The above list isn't definitive. I may find i need to create something else before i can get around to making one of those.

Anyway, here's the source and the iso image - feel free to comment, make suggestions, or whatever else!

Warning: Do NOT run this operating system on real hardware! Run it in a virtual machine, such as VirtualBox. Kernel level code can damage hardware components when not written properly. I am not responsible if you blow up your computer with my OS.
Attachments
my_os.zip
(127.29 KiB) Downloaded 91 times
¯\_(ツ)_/¯ It works on my machine...

User avatar
bad_brain
Site Owner
Site Owner
Posts: 11636
Joined: 06 Apr 2005, 16:00
19
Location: In your eye floaters.
Contact:

Re: I'm coding an operating system

Post by bad_brain »

good job so far buddy! =D>
I am really getting interested in this, I think mab will like it a lot too....years ago we both were considering building custom Linux distros but this way it is even more customizable... :D

will check it out on the weekend, will have to set up a VM first... *thumb*

P.S. how is the learning curve for this btw?
Image

User avatar
ayu
Staff
Staff
Posts: 8109
Joined: 27 Aug 2005, 16:00
18
Contact:

Re: I'm coding an operating system

Post by ayu »

I find this a very cool project! :)
"The best place to hide a tree, is in a forest"

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Thanks! I'm having a lot of fun with it.

The learning curve is (sometimes unfortunately) pretty steep. Additionally, there's limited information available about OS development in comparison to other programming topics, and not many people do this kind of thing, so the support base is a bit small. There are a handful of communities that revolve around os development that are willing to help answer questions at least.

The main difficulty with OS development is you're starting with absolutely nothing. No malloc, no printf, nothing at all. You have to write every function that you use. Also, you are directly interfacing with hardware, some of which is not open spec. For example, my vga driver writes directly to video memory to display text. However, i can't write a driver for my specific graphics card because i don't know how to interface with the hardware - there is no specification available for that. Luckily, i can write a VESA driver that gives high resolution and 32 bit color support - but no 2d or 3d acceleration.

Wow, this post got long fast... i guess i'll stop rambling, lol.
¯\_(ツ)_/¯ It works on my machine...

User avatar
maboroshi
Dr. Mab
Dr. Mab
Posts: 1624
Joined: 28 Aug 2005, 16:00
18

Re: I'm coding an operating system

Post by maboroshi »

Honestly way back in the day I had a dream to build an operating system using a high level language (Python).

As funny as that sounds people have done this.

and I am actually considering doing what these guys are doing as a starting point (and for the learning experience) because Gogeta is inspiring me with his Operating System

http://python-os.info/en/

Anyway

GREAT job man! :D

*cheers

Mabo

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Wow, i didn't know you could even build an OS in a language like python. I suppose it would be possible, if you wrote an interpreter for it in assembly, or c, or whatever.

It's pretty cool that they're doing it though ^_^

Anyway, i've updated the main post with my new release and a new screenshot. All i've added is support for interrupts, but that'll enable me to do a lot more now. I'm finding out that x86 computers are interrupt-driven systems ^_^

Next on my list is either gonna be keyboard input, a physical memory manager, or basic multi-threading support for the kernel.
¯\_(ツ)_/¯ It works on my machine...

pseudo_opcode
cyber messiah
cyber messiah
Posts: 1201
Joined: 30 Apr 2006, 16:00
17
Location: 127.0.0.1

Re: I'm coding an operating system

Post by pseudo_opcode »

Great job man...
I am jealous of you.. i know all of us have dreamt of writing our own OS at some point or the other.. and you're actually doing it..

Your intentions might not be to make it a state-of-the-art OS.. and that's how linux and ruby(though not an OS but language) started...

An idea would be to make it specialized for something.. that would be pretty awesome too.. i know b_b talked about the idea of specialized OS few years back :)

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Thanks! It's really frustrating and difficult at times. Mainly because it's so hard to debug code. I have a virtual machine with debugger built in, but it's all assembly code at that point... ](*,)

Yeah, i don't have any intentions for this OS but to learn from it, but who knows, maybe it will become something great! ^_^
I'm not sure what i would make the OS specialized in, though.

Anyway, new update!

I've implemented keyboard support. The code for the keyboard driver is kind of interesting, so i suggest checking it out ;)

There's no command line, so you're just typing to the screen buffer, but i'll add in a command line soon - it's next on my list.

New screenshot and binary/source uploaded.

Edit:

I forgot to mention, there's a bug in virtualbox (i think) where the caps lock key acts a little funny the first time you use it. When you hit caps lock and then start to type, it won't type the first character. After that, it works fine. I think it's a bug in virtualbox, because it doesn't do it in the other virtual machine i tried it in.

What it's doing is sending the scancode for the capslock key twice, the first time you do it. It keeps doing it until you press another key.

Also, there's a bug in my code somewhere, where when the kernel is loading up, it hangs occasionally. It's trying to execute code in an invalid code descriptor. I'm not sure why it's happening, but i'm working on it. If it hangs on you, just reset the virtual machine.
¯\_(ツ)_/¯ It works on my machine...

User avatar
maboroshi
Dr. Mab
Dr. Mab
Posts: 1624
Joined: 28 Aug 2005, 16:00
18

Re: I'm coding an operating system

Post by maboroshi »

I am really impressed with what you have done so far man.

Keep at it :D

pseudo_opcode
cyber messiah
cyber messiah
Posts: 1201
Joined: 30 Apr 2006, 16:00
17
Location: 127.0.0.1

Re: I'm coding an operating system

Post by pseudo_opcode »

Gogeta70 wrote:Yeah, i don't have any intentions for this OS but to learn from it, but who knows, maybe it will become something great! ^_^
I'm not sure what i would make the OS specialized in, though.
Yeah dont worry about making something out of it.. just do it coz you love it...

And yes.. i am impressed too.. and i am not easily impressed. And i know you're learning a great deal from this..

User avatar
maboroshi
Dr. Mab
Dr. Mab
Posts: 1624
Joined: 28 Aug 2005, 16:00
18

Re: I'm coding an operating system

Post by maboroshi »

In regards to the Python Operating System

https://github.com/jtauber/cleese/

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Yeah, that's why i'm doing it ^_^

I'm glad to see so much interest in my project! If you all have ideas for my OS, or questions, feel free to put them out there :D


I think building an OS in an interpreted language would be pretty cool, but my only concern would be the performance impact of having to translate all the code before it's executed - including most of the kernel itself... :-k

Anyway, i'm gonna build a quick-'n-dirty console for keyboard input, so i can implement commands that will help me with building the operating system. Then, i'm gonna get started on the physical memory manager. I'm looking at virtual memory management too, but that's gonna be a bit complex it seems - so it may take a while to implement.
¯\_(ツ)_/¯ It works on my machine...

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Hey guys, here's an update. No source code/screenshot with this one, though, unless someone really wants it.

Since the last update, i've made a console for inputting commands, and i've created a physical memory manager (aka a page frame allocator). The physical memory manager is pretty easy, it just keeps track of 4kb blocks of physical memory - handing out free blocks and marking them as used, or vise-versa.

Right now, i'm working on the virtual memory manager. Although, more specifically, i'm working on enabling paging. By doing this, i can isolate processes and the kernel into their own virtual address spaces. It's a bit tricky though, because once i enable paging, i have no direct access to physical memory. The processor translates assembly level addresses (ie mov eax, 0xDEADC0DE; call eax <-- 0xDEADC0DE is treated as a virtual address) into a physical address.

This means that i have to compile my kernel to load into physical memory at a certain address, but the code has to use virtual addresses for things like function calls - or anything that dereferences a pointer. Because of this, i have to write a special function that enables paging without calling any other functions.

So, how am i doing it? Well, here's my idea - and so far it's working.

My kernel is loaded to the 1mb physical memory mark - 0x100000.
The virtual address of my kernel is 0xC0000000 - the 3gb mark.
So, all my kernel functions are referenced at 0xC01xxxxx...

I write a function, say vmm_init(). In my kboot.asm (which is the code that is called by GRUB), i do the following:

Code: Select all

extern vmm_init
mov ebx, vmm_init
sub ebx, 0xC0000000
call ebx
This code allows me to call my function from it's real, physical address. However, now that i'm inside the function, i can only use local (stack) variables and no other functions unless i calculate their physical address.

From here, i set my page directory up - which has 1024 page tables. 1 Page table has 1024 page table entries that specify a 4kb aligned physical address, along with some access specifier bits (kernel/user access, read/write, execute, present/not present page, etc). A single page table represents 4mb of physical to virtual memory mapping. Since there are 1024 of these page tables, they represent, in total, 4gb of memory.

Since i'm working with physical addresses while setting this up, once i enable paging, i won't be able to refer to the page directory/page table structure to modify pages while the OS executes. How do i solve this? I map the last page directory entry to the address of the page directory - and now i know i the virtual address of the page directory ^_^

Anyway, as you can see, it gets a bit complicated. I'm having a lot of fun with it though :P

Sorry for the random "tutorial" lol
¯\_(ツ)_/¯ It works on my machine...

User avatar
bad_brain
Site Owner
Site Owner
Posts: 11636
Joined: 06 Apr 2005, 16:00
19
Location: In your eye floaters.
Contact:

Re: I'm coding an operating system

Post by bad_brain »

hm, so you can freely decide into which memory segment to load? would be interesting to know if there are differences in the performance of the segments (like on a HDD).... :-k
Image

User avatar
Gogeta70
^_^
^_^
Posts: 3275
Joined: 25 Jun 2005, 16:00
18

Re: I'm coding an operating system

Post by Gogeta70 »

Yes, you can load anywhere into physical memory. For example, my kernel resides at the physical memory address of 0x00100000 in RAM, which is the 1mb mark. However, i have mapped the virtual addresses 0xC0000000 - 0xC0800000 to physical addresses 0x00000000 - 0x00800000. Virtual addresses don't apply to hard drives though. I'm speaking strictly about RAM here.

Anyway, new update! I've successfully enabled virtual memory paging, and after a couple days of some extremely frustrating bug hunting, i got everything ironed out! In fact, the bug i was hunting down was the same one that was causing it to fail to boot sometimes before. It just reared it's ugly head when i enabled paging.

It turns out enabling interrupts after installing your interrupt handlers, but before redirecting IRQ's to the proper address in your interrupt descriptor table, will cause the system timer IRQ to generate a double fault exception... lol. Talk about frustrating.

New source code/binary uploaded. No new screenshot, as you can't really tell a difference between running in virtual memory and physical memory from a picture. Also, there's a console to play with. Currently, there are 2 commands: "help", and "memory," which lists available physical memory ranges along with the available memory in kilobytes.

I haven't decided what i'm going to work on next yet. What do you guys think i should implement next?
¯\_(ツ)_/¯ It works on my machine...

Post Reply