My 64 bit multiprocessor x86-native Operating System and related projects

email: info @ whoosh777 . com

A brand new modern fully 64 bit multiprocessor x86 native Operating system that I am designing and coding from scratch.

The OS is written in 64 bit x86 asm and boots directly from a floppy disk: insert floppy disk in PC, switch on, boot up. Hits the hardware directly (unhosted OS), doesnt depend on Linux or Windows. Stand-alone. feb.2007: the OS now boots on my laptop as well, the complicating factor being that laptops dont have standard floppy drives.

The idea is to create an OS which is as advanced as any mainstream OS yet also usable + fun. dissatisfaction with existing systems!

I am also creating from scratch my own 64 bit C compiler to use for and on the OS. But I have decided to develop my own language to develop the OS with, so am working on that now and will postpone further work on the C compiler till later.

Eventually the project will boot from hard disk and once the new language is ready I will use it to code OS components. The C compiler is being written entirely in C and will bootstrap from any existing C compiler on any OS. But everything now will revolve around the new language and not on C.

As of 2nd July 2007 the OS is multiprocessor pre-emptive multitasking, the multitasking subproject isnt yet complete.

BLURRY SCREENSHOT: first sector of the laptop HD echoed by my OS DMA HD code

Note 55 aa which is the last 2 bytes of an x86 HD sector 0.

DOWNLOAD VIDEO OF MY OS IN ACTION

The idea

The idea is that when the project is complete, the user will buy or build their own standard x86 PC. You then buy the OS from me which you then run on the machine.

Developers can develop their own programs to run on the OS. They can sell those programs to users or create free programs. I do not control what developers do.

And users can use programs written by developers, they either buy them or the developer gives the program for free.

Why not use Windows, AmigaOS or Linux instead?

Exactly! That is the challenge, to make the system more interesting for developers and users. So you would use my system instead of other systems because it is BETTER!

Same reason you would switch to a car which was cheaper, faster, safer and more comfortable.

Exactly how it is better mainly wont be discussed till much later on. But if you contact me by email I can explain SOME of the ways it is good. I guarantee that when you use the system you will quit whatever system you currently use except for things not implemented yet. How can I be so sure? well, I have used 3 of the mainstream systems: Linux, AmigaOS and Windows XP. So I have a pretty good idea of what the competition is. Windows probably impresses users, but I am not a user I am completing right now an entire kernel in x86 asm which is completely my own design. I designed and coded the entire system.

Example features are Unix quality security: immunity to viruses, hackers, spyware. Unix quality safety: eg protection from crashes. System architecture as elegant and direct as AmigaOS. In many ways MORE direct than AmigaOS.

The actual architecture is completely new eg immunity to viruses is being done completely my own way and the system architecture is completely different from AmigaOS.

Unix and AmigaOS are the 2 main influences, Unix tends to be definitive in terms of functionality but the implementation is an enormous inextricable hack. And AmigaOS tends to be definitive in terms of implementation quality but has problems achieving the functionality of Unix. Windows is also an influence on many specifics especially when they avoid unnecessary functionality.

The system will also be ideal for hosted OS's or emulators, hostability is a design constraint.

I feel I can create something better than what is on offer. Microsoft are moving forwards unstoppably and a countersystem is urgently required.

[PROJECT SUMMARY on 25th November 2006]

Project progress

In reverse chronological order:

12.oct.2013[ OS and language projects continue...],

13.feb.2011[ OS project survey continues, good progress ],

3.feb.2011[ EXTREMELY BUSY reviewing the OS work ],

27.jan.2011[ OS PROJECT RESTARTED! ],

26.jan.2011[ INITIAL COMPILER PROJECT IS COMPLETE! (other than testing)],

2.dec.2010[ COMPILER PROJECT IS "COMPLETE"! ],

23.nov.2010[ version 0 is built, and various bugs resolved ],

21.nov.2010[ version 0 of language is written! ],

2.oct.2010[ version -1 of language complete! ],

1.july.2010[ working on floating points ],

21.may.2010[ language almost complete, just some floating point stuff left to do ],

14.sep.2009[ language visibility complete ],

2.sep.2009[ LAST TRICKY COMPILER CODING COMPLETE! ],

15.aug.2009[ a lot of object implementation complete ],

8.aug.2009[ major component of objects complete ],

29.july.2009[ implementing objects ],

6-8.july.2009[ initial general purpose language done, last remaining features ],

30.jun.2009[ the language is now general purpose ],

17.jun.2009[ one feature left! ],

2.jun.2009[ implementation of if-else REALLY complete! ],

28.may.2009[ implementation of if-else "complete" ],

21.may.2009[ several subphases of if-else "complete" ],

1.may.2009[ current switches complete ],

30.apr.2009[ further switch statement features complete ],

28.apr.2009[ various switch statement features complete ],

25.apr.2009[ on last subphases of implementing switch statements ],

25.jan.2009[ complex source files compiling correctly ],

23.jan.2009[ initial limited version of compiler is written ],

10.jan.2009[ holiday, work continues. ],

27.nov.2008[ excellent progress, plenty left to do ],

20.nov.2008[ compiler project last phase progressing ],

11.nov.2008[ working on last phase of compiler project ],

9.nov.2008[ completed SECOND LAST phase of compiler project! ],

25.oct.2008[ compiler phase "complete", 2 phases left ],

3.oct.2008[ new compiler phase, 2 phases left ],

26.sep.2008[ holiday, tricky compiler coding ],

21.aug.2008[ OS modified for compiler ],

19.aug.2008[ OS now fine on Sempron ],

14.aug.2008[ restarts OS project for compiler ],

6.aug.2008[ optimisation, OS + compiler + language, ],

3.aug.2008[ About to start on LAST difficult subproject of compiler ],

1.aug.2008[ compiler now 64 bit Linux Ubuntu and Fedora hosted, HUGE BUG RESOLVED ],

24.july.2008[ compiler now also Linux hosted, much progress ],

18.july.2008[ current code compiled, bug ],

15.july.2008[ things get very complicated ],

9.july.2008[ sizeof equivalent completed ],

3.july.2008[ restarting work ],

18.jun.2008[ sizeof equivalent: component completed ],

17.jun.2008[ sizeof equivalent ],

13.jun.2008[ much work, slow progress ],

11.jun.2008[ register usage decisions ],

9.jun.2008[ working on function calls ],

5.jun.2008[ compiler project: things get very complicated ],

4.jun.2008[ function call register usage ],

25.may.2008[ literal integer consts ],

24.may.2008[ implementing varargs and optional args ],

24.apr.2008[ changes to the type system implemented ],

23.apr.2008[ implementing changes to the type system ],

18.apr.2008[ type assignability new problems resolved ],

17.apr.2008[ type assignability new problems ],

16.apr.2008[ type assignability difficult subphase coded ],

11.apr.2008[ continuing with code generation semantics, type assignability ],

9.apr.2008[ type equality "complete" (for now) ],

8.apr.2008[ type equality some major features complete, further bugs ],

7.apr.2008[ type equality implemented ... bugs ],

5.apr.2008[ type equality: steady progress ],

3.apr.2008[ type equality very very tricky! ],

30.mar.2008[ type equality ],

22.feb.2008[ progressing towards code generation of compiler ],

19.feb.2008[ first version of major compiler component now functioning ],

2.feb.2008[ starting to make more progress, ],

17.jan.2008[ difficult language semantic design, ],

13.jan.2008[ continuing implementing type checking, ],

10.jan.2008[ holiday complete, so much to do. ],

3.dec.2007[ continuing working on type checking ],

18.nov.2007[ very difficult type coding ],

16.nov.2007[ implementing type checking ],

15.nov.2007[ plenty of work, gradual progress ],

11.nov.2007[ the work intensifies ],

9.nov.2007[ very busy implementing language semantics ],

8.nov.2007[ gradually progressing with second phase of language ],

7.nov.2007[ have started second phase of language ],

6.nov.2007[ transition phase complete and untested, bugs, changes ],

4.nov.2007[ first phase of my language complete, transition phase ],

3.nov.2007[ almost completed this first phase ],

31.oct.2007[ difficult design decisions ],

29.oct.2007[ Almost ready to start next phase of my language ],

27.oct.2007[ Progressing with my own language ],

26.oct.2007[ total change of plan? ],

25.oct.2007[ first version of current C++ phase complete but not usable yet, ],

24.oct.2007[ progressing with current C++ phase, different from Stroustrup ],

22.oct.2007[ tricky C++ ],

19.oct.2007[ C++ inital subphase done ],

18.oct.2007[ C++ inital subphase almost done ],

17.oct.2007[ busy working on C++ ],

16.oct.2007[ C++ subproject restarted. ],

14.oct.2007[ first version of C- complete ],

13.oct.2007[ C- "using" bug resolved ],

12.oct.2007[ C- "using" directive ],

11.oct.2007[ C- templates: debugging ],

08.oct.2007[ C- templates: trickiest work now done (I think) ],

06.oct.2007[ C- templates: ever more complicated. ],

05.oct.2007[ working on C- templates. ],

16.sep.2007[ C- mostly done, 1 tricky thing to do ],

12.sep.2007[ a bit of progress towards C-. ],

10.sep.2007[ work towards C- very tricky. ],

5.sep.2007[ gcc compatibility is painful! ],

4.sep.2007[ C- and preprocessor. freeze up bug. resolved. ],

3.sep.2007[ C- very deep bug ],

2.sep.2007[ many difficult problems. C- first phase initial version ],

21.aug.2007[ trickiest ],

20.aug.2007[ C- ],

19.aug.2007[ x (y) , an idea ],

18.aug.2007[ further C and C++ ambiguities ],

17.aug.2007[ continuing with the initial C++ work, ambiguities ],

15.aug.2007[ some very initial C++ done ],

13.aug.2007[ pre-processor done for now. coding C++ now. ],

11.aug.2007[ oh thank you gcc! ],

10.aug.2007[ pre-processor ],

9.aug.2007[ towards C++, pre-processor ],

2.aug.2007[ restarting compiler project, C++ ],

1.aug.2007[ WHEEEHOOOO! ],

31.july.2007[ slow progress with bugs ],

30.july.2007[ difficult multiprocessor bugs ],

29.july.2007[ much work, bugs progress ],

9.july.2007[ major improvements underway ],

8.july.2007[ multiprocessor semaphored text now good, video thereof ],

7.july.2007[ many problems, gradual progress ],

6.july.2007[ found a bug ; gruntwork ; strange + interesting problems. ],

5.july.2007[ pre-emptive multiprocessor multitasking seems ok now, video ],

4.july.2007[ pre-emptive multiprocessor multitasking debugging continues ],

3.july.2007[ very tricky work towards pre-emptive multiprocessor multitasking ],

2.july.2007[ first version of 64 bit x86 PRE-EMPTIVE MULTITASKING functioning! ],

17.jun.2007[ another subproject completed. continuing, ],

16.jun.2007[ working on major bug, resolving ],

15.jun.2007[ difficult bug in subsystem ],

14.jun.2007[ working on everything now towards multitasking ],

13.jun.2007[ some architectural subphases towards multitasking completed ],

12.jun.2007[ an impossible bug, 315am resolved ],

10.jun.2007[ working on everything now ],

8.jun.2007[ up to date, about to begin major subphase to multitasking ],

4-5.jun.2007[ working on the earlier code ],

2.jun.2007[ major phase completed towards multitasking. continuing from where I left off ],

31.may.2007[ progress on absolute time ],

25.may.2007[ time progress ],

24.may.2007[ laptop time workaround, further time problems ],

23.may.2007[ OS time delays, laptop time problems ],

21-22.may.2007[ a lot of technical work, delay functionality ],

13.may.2007[ much progress on multitasking components ],

16.apr.2007[ impossible problems ],

15.apr.2007[ difficult to see bugs, complexity meltdown ],

14.apr.2007[ further semaphore work ],

13.apr.2007[ a lot of progress towards semaphores ],

11.apr.2007[ very busy ],

09.apr.2007[ one component complete ],

06.apr.2007[ very gradual progress ],

01.apr.2007[ all multitasking problems combine ],

31.mar.2007[ semaphores: stressful and interesting, ],

30.mar.2007[ many theoretical problems, semaphores... ],

29.mar.2007[ memory architecture work continues ],

28.mar.2007[ very difficult multitasking component ],

25.mar.2007[ redone yesterdays memory work ],

24.mar.2007[ debugging code so far, improved memory ],

20.mar.2007[ centre of the OS, ],

18.mar.2007[ the work continues, OS photo ],

15.mar.2007[ working on an early phase towards multitasking ],

13.mar.2007[ work continues towards multitasking, ],

9.mar.2007[ a batch of OS code done, ],

6.mar.2007[ much coding, PATA independent of BIOS, ],

5.mar.2007[ 28-bit 2006 ],

4.mar.2007[ revisiting PATA code ],

2.mar.2007[ fork()?, fantastic bug! ],

28.feb.2007[ low level hardware coding, ],

26-27.feb.2007[ some very initial code functioning ],

22-25.feb.2007[ very busy on the project, bug ],

21.feb.2007[ gradual hardware progress, ],

18.feb.2007[ project fine on tower system, work continues ],

16.feb.2007[ HUGE PROGRESS, all bugs resolved, project ported to laptop ],

15-16.jan.2007[ supervisor level bug progress, ],

14.jan.2007[ restart on supervisor level bug, ],

29.nov.2006[ further progress on supervisor level bug, ],

28.nov.2006[ compiler enhancements, supervisor bug work continues,],

26.nov.2006[ some progress on supervisor bug ], 25.nov.2006[ C preprocessor complete. OS kernel work continues ], 21.september.2006[ low level OS component complete, new chapter of project]

21.sep.2006[PROJECT SUMMARY]

Click for Ghostscript project

I am creating my own OS which will run on sufficiently modern standard x86 hardware. Exactly how modern varies a lot. eg I will support the PS/2 mouse which is quite ancient. However I wont support CHS HD sector addressing which is relatively recent. This is not an encyclopaedic project, it is pragmatic.

The OS runs directly on x86 hardware and isnt hosted. The current work is entirely written in x86 asm. There is a huge amount of work to create just the beginnings of an OS. The logistics are very complex even to do something trivial: the most trivial of actions usually occur in a context and for an OS project you'd need to create said context. so eg a no-op program:

int main( int argc, char **argv ){return(0);}

needs a C compiler. It needs some scheme for storing the binary on disk, not necessarily a filesystem. It needs a program loader, it needs a memory system to organise space for the program to run in. If the OS is multitasking then it needs task management. Even if you use gcc for the C compiler which I dont you'll have to create an ELF loader which is a moderate amount of gruntwork. I have written my own ELF loader but I wont use it.

But I am making very good progress. See the first project update for some of the things I've got running.

As a major subproject I am creating my own 64 bit C compiler to use instead of gcc. This will be used later to bring my FS project to my OS project. 18th April 2008: in fact I am now developing my own new language from scratch and a compiler for it. That is moving steadily towards completion,

There are 4 flagship subprojects: C compiler, FS, OS and new language + compiler. The FS will be cross compiled to the OS by the compiler, and the compiler will run above the FS in the OS. And the OS will boot eventually from the FS. Currently the OS boots from a floppy disk without FS. Once the compiler for the new language is complete future components of the OS will be coded using the new language.

I have made major progress on the bootstrap towards a very modern brand new OS. I am designing and coding the entire system myself. There is no team, I am creating everything whether it be HD drivers or FS or compiler or shell or graphics system. And I am designing the entire architecture myself including the memory architecture, the program environment, task protection, drivers, FS architecture etc.

Nothing is reimplemented, nothing is ported, nothing delegated, no teams, total attention. Completely brand new and sophisticated. total design: I design every aspect of every thing, this makes a big difference to quality as it generates a very high level of design harmony.

There is an enormous amount of gruntwork before I can begin on the interesting and fun things. I think I am reaching the end of the most severe part of the project.

With the current work you insert the boot floppy disk in drive A: of a PC, then you switch the PC on and it boots from that. You dont need any existing OS and in fact you dont even need any hard disks. The disk then boots up. As it boots it echoes to the screen what it is doing. eg in the middle of the bootstrap the current binary initialises the IDE drives and echoes what the drives are: if there are none it will echo this. If you have just 1 HD it could be on either cable. The PS2 mouse is initialised later. The compiler for the new language is now at the semantic processing phase, eg I have already implemented type equality checking and am currently working on type assignability (18th April 2008).

At the moment I can write and run programs utilising various hardware by appending the program to the end of the bootstrap. I dont yet have a proper program architecture but can nonetheless run programs. so I can write programs currently that eg read HDs and read PS/2 mouse events via a low level API. Basically the code I have is a rudimentary kernel but not yet an OS.

A lot of the work is implementation side and technical. eg I need more than one ABI as x86 is basically a glob of different CPUs. Creating an ABI takes some work. x86 has various complicating factors eg a near call has a different stack frame from a far call. Its not like 68k where the stack frame is always just the 32 bit return address followed by the pushed arguments. With 16 bit x86 you cannot have more than 65536 bytes of stack: you could in theory hack beyond 65536 via some form of stack switch but that would have to be done in software.

Contrary to popular belief little endian memory is better than big endian memory: little endian is forwards compatible with increased field widths which big endian isnt. If you look up about endianess on the internet you will always be told that both schemes are equally good: WRONG! eg with little endian you can do:

void bit_set( void *x, int bit )
{
int *X = x ;

X[ bit/32 ] |= (1 < < (bit % 32) ) ;
}

which is mathematically clean and will set ANY bit of ANY entity, eg bit 1000 of a 65536 bit entity. eg it will set bit 50 of a quad or bit 18 of an int or bit 13 of a short or bit 7 of a char. This function is IMPOSSIBLE with big endian as big endian points to the wrong end of an entity making bit position indeterminate.

Basically you can only do the above with little endian as it is mathematically correct.

Little endian is better as the address of an entity is always the address of bit 0, which is mathematically sound. eg you can do:

int *x ; short *y = (short *)x ; char *z = (char *)x ;

which would be a bug with big endian. little endian has better structure than big endian. It can do more things and is more efficient.

whereas big endian crazily gives the address of a short as the address of bit 15, the address of an int as address of bit 31, address of a quad as address of bit 63. So big endian is mathematically unsound. This is one of the reasons why x86 has lasted so long since 1980: it is easier to extend the CPU definition and easier to extend OS definitions.

Say you have 12345678 in memory, big endian says the address of this is the address of 12. But as a word the address is the address of 56 and as a char the address is the address of 78. With little endian addressing the address is always the address of 78 regardless of whether it is a char, short or int. big endian is like having the post box of a building always on the top floor and numbering the floors from the top downwards! eg:

floor1
floor2
floor3
....

only useful for helicopters and addressing problems if you want to make the building higher.

Many people believe x86 has a problem by being little endian: FALSE! 68k has the problem by being big endian. eg there are various bugs which only happen on big endian systems. I think PPC is dual endian ie it can be configured either way but it tends to be configured as big endian. I think if I ever ported my project to PPC I would configure PPC to be little endian. Its some years since I looked this up but IIRC PPC has a CPU config bit somewhere which makes it little endian, I know MIPS has such a config bit and PPC is very similar to MIPS. You'd have to procede carefully as any BE BIOS would now malfunction and bytes may need reversing for hardware registers: comparable problems happen on x86 where the BIOS will malfunction once you start reconfiguring the CPU.

I have made such fantastic progress with x86 that I'd be mad to switch to anything else. x86 is a very Darwinian scene. Anyone can participate, and if you can outdo the others you will be rewarded for it. If a company in this scene stops progressing another company will soon take its place. It means if you involve yourself with x86 and you make an effort success is more or less guaranteed.

My OS also will be presented according to Darwinian principles, so if you are a programmer and know your art, you'll make plenty of money. This is very much a programmers OS.

==== Projects progress ====

My filesystem + OS projects progress will both be here. Times may be out by ± ½ hour.

============== 12th oct 2013 ===============

More than 2 years since the last update!

I have made some major progress with the OS project since then, which now uses modern pixel graphics. But there is a lot of work to be done.

Although the language project is complete, I have decided to make some changes to the project, and these have been stalling progress.

In recent times I have had to spend a lot of time on non computing things which has also slowed down progress. But I have more time available now and hope to start progressing again.

============== 13th Feb 2011 ===============

The survey of the OS project continues, but I'm making good progress. So far the work is entirely in asm.

Later on I should be able to upload some new videos of the OS functioning.

============== 3rd Feb 2011 ===============

I'm extremely busy reviewing the OS work. I'm also making a lot of small enhancements. It will be a lot of further work before I can start trying to integrate the compiler with the OS. There are a lot of more basic problems I need to work on first. I'm only doing a superficial review as the totality would take a long time to study, eg the ATA code would be a lot of work to verify.

Anyway its MUCH MORE interesting than working on the language,

I had problems with the keyboard and mouse with the laptop, eventually I got the keyboard and mouse functioning perfectly with the laptop, but there is something wierd going on.

============== 27th Jan 2011 ===============

OS PROJECT RESTARTED!

I will continue testing the compiler project in parallel to this, but I decided to restart the OS project right away.

The first problem is to try to build the OS where I left off.

To do a full rebuild and to then try to run the OS.

The OS is built on Windows XP using Windows BIOS programs written on WinUAE and then built from the XP shell. The assembler uses Intel syntax, which is slightly different from AMD syntax. When I last built the OS I used AMD syntax with Intel incompatible stuff done in macros, but at some point I redid things with Intel syntax. But now the assembler wont assemble because it has bugs! 1 version of the assembler has wrongly implemented one Intel opcode and another version has wrongly implemented another. Hopefully the latest version of the assembler will work...

============== 26th Jan 2011 ===============

THE INITIAL COMPILER PROJECT IS COMPLETE, other than testing the code.

I have implemented the auxiliary functions, and also have made many technical enhancements.

The work at 2nd December 2010 didnt have satisfactory object orientation, I completely redid the implementation of the object orientation as the existing implementation didnt deal with various technicalities and was too complicated to study.

The language definition of objects hasnt changed, just the implementation. Objects are very complicated to implement.

I decided to change the ABI for register args of functions, I have just now completed implementing the new ABI which is much better.

ALL intended work for the initial version of the language is now complete,

There will be ongoing technical enhancements but the language is now ready for working on the OS. I am ready to restart the OS project.

============== 2nd Dec 2010 ===============

COMPILER PROJECT IS "COMPLETE".

The core compiler project is "complete". I still have some auxiliary functions to code, but these are external to the compiler and more like a compiler library.

When I say "complete", I mean version 0 of the language, which is the form necessary for coding the OS.

There are of course further things that need implementing, but these are nonessential, and I need to do a lot of further testing.

The compiler project has thus taken 3 years, 1 month and some days!

It should have been taken much less time, perhaps 2 years, but this was due to lack of discipline and continual distractions.

But I am now super focussed, and in fact am working much more productively than in say 2007.

The project was starting to take forever.

I hope to restart the OS work REALLY soon.

============== 23rd Nov 2010 ===============

Is it really 12 days since version 0 was written!

I have meanwhile built the compiler, and have been testing and debugging the code.

There were some quite subtle bugs, I had commented out something in 2008, and now realized the original statement was in fact correct. This was to do with the implementation of function calls. I usually put the date and time when I comment out stuff, as I find it fascinating to look at the datestamps some years later.

There were continual bugs for some object oriented example, with some optimization to evade an extra register move. I decided to remove the optimisation and immediately the bugs all flew away! The optimisation had been a perennial nuisance.

Much more maintainable without the optimisation. The code is slightly less efficient but its better without the optimisation.

I need to test some further fundamentals, then a small amount of minor enhancements and the project is complete!

============== 11th Nov 2010 ===============

I have completed implementing version 0 of my language. This is the final form of the language. I havent built the code yet, and I will also need to test and debug this code.

This form of the language will be used to implement the further phases of my OS.

It has taken a bit over 3 years to complete the language, which is much too much time.

Although there is further work, the project is kind of complete.

I had to do some extremely tricky implementation of certain object orientation features since version -1.

Certain auxiliary functions arent implemented yet, but these are external to the language and compiler.

I should very soon be able to at last restart the OS project,

============== 2nd Oct 2010 ===============

Version -1 of my language is complete. There are some library functions I havent yet implemented. But they are external to the compiler and language definition.

ie the language can now be used, but for some things I have to implement some library functions.

I want to make one change to the object orientation before its version 0. I will then test that very thoroughly, and once tested it will be version 1.

I hope VERY SOON to continue with the OS project.

I am at last progressing with the core projects.

It will be some work before I can use the language from the OS as the language uses a different ABI from the OS. I have to figure out how to interface the 2 ABI's.

The OS so far uses an ABI good for hand coding, but the language uses an ABI good for compiler output.

So the initial testing is actually manual, I read the output asm and verify it by reading, not by using the code. But eventually I will use the code and the first project is to create modern graphics for my OS.

The graphics will use the CPU as a GPU,

============== 1st July 2010 ===============

The ICC colour profile stuff is complete, and some of the code can be downloaded from the homepage. That has nothing to do with the OS and language projects, just mentioning it as I am the same person working on different projects not just those on this page. And as I work on one project I am not working on the others.

Working on the compiler again. I am working on the floating point functionality, which is proving to be extremely difficult. But I am progressing with this.

============== 21st May 2010 ===============

No I havent quit! But I have been doing too many different things, eg I have been working on my own code to print with the Samsung CLP-315 printer, and on coding tiff files and next I am working on ICC colour profiles, which is not related to the projects on this webpage but the other webpage.

I have made a lot of progress on the language since the last update, all that is left to implement is some floating point functionality. Everything else is complete.

In fact I could start using the language already, but I decided I would complete the floating point functionality first. Either way I need to also test the compiler very thoroughly.

I have also gotten distracted by too many other things. But I am trying to get back to the core projects of the compiler and the OS.

I dont have the time left to make regular project updates, thus there will be less updates here. But my plans are unchanged, its just that a lot of other plans have emerged which has led to some time mismanagement and time wastage.

============== 14th Sep 2009 ===============

I have completed implementing the language visibility. I have done a reasonable amount of testing of the code also and it seems fine. I will do more thorough testing right at the end of the project.

I have managed to implement ALL visibility features that I wanted to implement.

============== 2nd Sep 2009 ===============

I have completed today the LAST tricky coding of the compiler and language project. There is a lot left to do but its all straightforward work.

I have decided to complete all the core language features before stabilising the work.

Enhanced conventional objects are now implemented, but there are some further straightforward object features I want to implement.

I hope now to make major progress towards completing the project. But all the ideas are paying off now.

============== 15th August 2009 ===============

I have completed writing, building and debugging a majur amount of the object code.

The compiler now compiles a lot of object functionality correctly, there were a lot of bugs initially. Its really nice seeing high level source code compiling correctly.

There is one last further batch of very complicated coding to do. I am optimistic that I can complete the implementation of objects this month.

============== 8th August 2009 ===============

I have completed one of the really difficult components of the enhanced conventional object system.

There are at least 2 further tricky components. At the moment I am working on another component.

I think it is feasible to complete the main implementation of conventionl objects this month.

============== 29th July 2009 ===============

I am working now on the LAST tricky topic which is conventional objects, plus various language specific extensions.

conventional objects are extremely tricky to implement especially the language specific extensions. conventional objects introduce various new problem phenomena.

In fact the conventional objects are highly unconventional!

They are conventional in the sense that you can do EQUIVALENTS of objects of other languages, but unconventional in the way you do things.

The unconventional design is extremely efficient, but also extremely difficult to implement. It is VERY difficult to get right.

But I am gradually progressing with this.

I completed the main implementation of the enum mechanism a few weeks ago.

============== 6-8th July 2009 ===============

I completed superficial debugging of the initial general purpose language on 6th July.

I am now implementing the last remaining features of the initial project.

The initial project completion will be a language with the features I want for developing the OS.

The language right now doesnt have all the features yet, eg it doesnt have enums and objects yet.

Last night I completed implementing the initial enum features, today I will be working on one of the trickier features.

The work is now finishing off the project, but its tricky all the way to the end eg completing the implementation of objects is quite tricky.

============== 30th June 2009 ===============

with 30 minutes of June 2009 left I have completed enough features to make the language general purpose.

I havent checked or compiled the new code, and there are various further features before the initial project is complete.

============== 17th June 2009 ===============

I have just completed a major feature set, ONE FEATURE LEFT to complete the initial language!

============== 2nd June 2009 ===============

I ran into a lot of problems with if-else, and have been working on these.

if-else is now written built and relatively complicated examples are now compiling correctly.

This is a MAJOR completion point of the project.

============== 28th May 2009 ===============

The implementation of if-else is now written, built and simple examples are now compiling correctly. Further testing is necessary of more complex examples.

Also now implemented are various other features associated with if-else such as "or", "and", comparisons. The way the language does things is quite different from C.

Once the further testing is done this is A MAJOR completion point of the language and compiler.

The initial form of the project is virtually complete, just a few features left to implement.

The initial form of the language and compiler project will be sufficient to develop the rest of the operating system with. But there are further features beyond the initial form.

The compiler project has taken a bit too long, the biggest error was to attempt to implement C++. Thankfully I shelved that and have gotten on with something much more interesting and useful.

============== 21st May 2009 ===============

I have got several phases of implementing if-else written and compiled, not tested yet.

2 subphases left to write.

Once if-else is completed then the language+compiler project is moving to completion.

============== 1st May 2009 ===============

I have completed implementing the current form of switch statements.

Also I have resolved the bug with the switch error messages, this was a straightforward memory trashing bug:

for( j = 0 ; j < i ; j++ ){ .... ; mem[ j-1 ] = .... ; .... }

which trashes mem[-1],

once I corrected that, the bugs ALL vanished.

Anyway that is a MAJOR completion point,

There is at least one further switch feature I plan to implement later for the initial completion of the language.

============== 30th April 2009 ===============

I have got further of the current switch statement features now complete, functioning correctly and efficiently.

There is one further major feature I need to check, and then the current switch statements are complete.

There are some switch features I havent implemented yet, but will leave till later on. I think they will be unproblematic to implement.

There are some bugs currently with the response to certain errors, I will try to resolve those error response bugs. I will rebuild on Linux to locate those bugs.

on AmigaOS they result in a Geekgadgets error message requester.

============== 28th April 2009 ===============

various major switch features are now fully implemented.

I managed to successfully compile some switch statements of the language to asm today, and the output asm was correct: took some debug work to get correct output. But I managed to do all the debug on AmigaOS without needing to use Linux. The Linux build is necessary for trickier debug, I will test the switch statements on Linux later as Linux is particularly good at locating bugs.

That is major progress, that various switch statements are now complete.

I will test out other features later today.

============== 25th April 2009 ===============

4 months since the last update! I have been extremely busy working on the language.

I am working on the last subphases of implementing switch statements for the language.

Progress should start accelerating.

============== 25th Jan 2009 ===============

There was a bit of work to build the compiler. Then on simple examples I located a straightforward bug where register restore was in fact saving registers. just a typographical error.

The initial limited version of the compiler is now compiling simple source files correctly to 64 bit x86 asm. I am about to now try out more complex examples.

Once this has stabilised this is a major completion point of the project. All the remaining work is much easier. There is still a lot to do, but progress now should be very fast.

11pm: more complex source files are compiling correctly eg expressions such as f( f( a, a ) / f( b, b) , f( a, b) * f( b, a ) ) and f( f( f( a, a ), f( b, b ) ) , f( f( b, a ) , f( a, b ) ) )

============== 23rd Jan 2009 ===============

I have just completed writing the initial limited version of the compiler. As said in the previous update I would have completed it in December had I not taken a complete holiday from computing.

Not yet compiled or tested. Once it has stabilised I will then begin complete the other features of the language.

============== 10th Jan 2009 ===============

I was working towards the initial limited version of the language till the middle of December and took a holiday from programming till now. With that programming I made major progress towards the initial limited version of the language. The compiler already outputs 64 bit x86 asm. I think I could have got the initial version functioning within December but decided to take a holiday from programming and spend the time with my relatives instead.

The asm output already deals with eg register function arguments and implementing local variables via registers. It deals with nested function calls eg f( g( x, y ), h( z, t ) ) where evaluating an argument can trash the registers for the earlier args.

The asm output isnt yet usable as there are some further things to implement eg saving registers at the start of a function and restoring them at the end.

As soon as the asm output is usable then the initial limited version of the language is complete. If I can complete that by 25th January then I would have completed it in December had I not taken a holiday!

============== 27th Nov 2008 ===============

I am making excellent progress with the last phase of the compiler project. But there is a lot left to do. I have to keep moving back the timeline for this!

The work is quite tricky but I am progressing, and dealing with many long standing problems.

The aim now is to complete the initial limited form of the compiler by the end of December.

At the moment I am implementing the stack memory usage, and have been working on this since Tuesday. It is quite a complex problem eg dealing with var args.

The work is very interesting now

============== 20th Nov 2008 ===============

Progress continues on the last phase of the compiler. Various things are now complete and ready to be tried out.

There is problem after problem, but I am making progress towards the initial limited form of the compiler.

On 9th Nov I said I would have the initial limited form of the language implemented in a few days, it is proving to be a lot longer than that! more like a few weeks.

There is now already quite a bit of x86 asm output.

Progress now with the compiler project will greatly speed up. But the huge amount of things still to be done mean it could be a few months before the first true version of the language is functioning.

The work now is very interesting but also difficult, many of the problems are how to deal with extremal situations. The language is very different from C,

Probably I can have an initial form of the compiler functioning by the end of this month.

============== 11th Nov 2008 ===============

I am working on the last phase of the compiler project, where the binary is constructed. This is proving more complicated than I suggested in the previous update and will take longer than envisaged.

There are endless complicating factors, but I am working on these and making gradual progress.

The aim now is to get the limited form of the language functioning within 2008.

The difficulties relate to various language features, interesting new features which are difficult to implement. If I do away with such features it will be easier to implement, but I need to implement them to make the project worthwhile.

This last phase is where the entire project converges, there are some incoherencies between the phases where one phase conflicts with another. I am trying to resolve these.

============== 9th Nov 2008 ===============

I have "completed" another phase of the compiler project. I havent compiled it yet, I will get the code to compile next but wont try it out till the next phase is complete.

The next phase is the LAST phase of the compiler-language project and is relatively straightforward.

However all the latter phases will need further work, but the current work is enough to get a limited form of the language and compiler functioning. This initial limited form of the program wont be acceptable for programmers,

I will then do further work on the latter phases building up to the first true version of the language and compiler which WILL be acceptable for programmers.

Thus in a few days I HOPE to have the initial limited form of the project complete. But it is likely to be some weeks before the initial TRUE version of the language is ready.

I will see if I can get the initial TRUE version complete this year, but that is a challenge!

============== 25th October 2008 ===============

I have "completed" the current phase of the compiler.

It is compiled + tested, but I cannot test it thoroughly till the next 2 phases are completed. Linux debug is not detecting any problems.

I am about to start the second last phase of the compiler.

Once all the phases are "complete", there is further work with all the latter phases before the compiler is the first prototype.

I say "complete" because its not truly complete, however in terms of difficulty it is complete!

============== 3rd October 2008 ===============

I am already working now on the next phase of the compiler.

2 phases left and the compiler will start compiling.

There will be further work before the initial project is complete.

If I hadnt wasted so much time on C and C++ the compiler project would already be complete.

I am confident I can complete the initial version of the compiler this year.

============== 26th September 2008 ===============

I've been on holiday recently so not too much done. Right now I've restarted work on the compiler.

I've completed some further tricky work relating to function calls, eg things like f( g( a, b) , h( c ) * i( j( d ) + k( e ) ) )

The work was towards the compilation of such expressions. The arguments are typically registers, and I am trying to compile such things efficiently. Here if we set up the one argument in a register the other argument will typically trash the register.

Right now I am beginning work on the next phase of compilation. This is very very complicated. I am trying to figure out a plan for the work. I am making gradual progress.

============== 21th August 2008 ===============

I have completed some modifications of the OS to make it compatible with the compiler. I can now continue the compiler project. The forthcoming compiler output will be usable now within the OS but it will require stubs also.

The OS so far is ENTIRELY written in asm and uses a completely different ABI from the compiler. In fact the OS itself needs several ABIs as some things just have to be done differently from other things.

Stubs will be used to interface between mechanisms, trying to have the same ABI for everything will be too complicated.

Writing the OS entirely in asm was a good idea as I now have a thorough understanding of x86 which I can use for implementing the compiler.

Some of the asm will be gradually migrated to my own language.

============== 19th August 2008 ===============

After many days of effort I have the OS now functioning fine on both the uniprocessor Sempron and the dual processor Turion X2. both by AMD.

The bug was that PC's can disable bit 20 of addresses!

That means when you access an address Addr, the h/w clears bit 20 before reading/writing the address.

But the MMU is accessing Addr. An OS needs to re-enable bit 20.

Once I re-enabled bit 20 everything is now fine.

The OS project and compiler project are now almost ready to be merged.

============== 14th August 2008 ===============

I havent done any work with the OS for literally a year, if you look at the update summaries at the top.

I decided to rebuild the OS, I had forgotten even the build procedure.

Eventually I got the build functioning, it is an interactive build, as the build proceeds I have to modify the source each time: workaround of limits of the assembler. everything fine with the dual core AMD. However the uniprocessor AMD was crashing.

After some days of work I located the problem, reloading the MMU table was crashing the Sempron but NOT crashing the Turion X2.

The new MMU table had identical entries to the original, I verified that and it was correctly aligned. If I reloaded the original table no crash. Didnt make sense, but it was about 3MB into memory. PC's begin with just 1MB of memory, the initial MMU table will always be in the first 1MB. Perhaps NOONE has tested the Sempron with a MMU table outside the first 1MB!

I carefully arranged for the new MMU table to be within the first 1MB, and no more crashes. Looks like its a CPU bug with the Sempron. Thats the problem with severely complicated CPUs such as x86 that it becomes impossible to really thoroughly test the CPU.

I could get around the problem by copying new MMU tables to existing ones but that will be very slow. Instead for the obsolete Sempron I am currently limiting the number of MMU tables to 16 which I can fit in the first 1MB. That will allow me to test code on the Sempron, but my OS will be meant for machines after the release of Vista.

No more crashes with this change, the Turion X2 dual core functions fine just like in the video I made at the time, but the Sempron tower isnt quite right, it doesnt seem to ever start echoing the 111[ 0 ] things. Probably a software bug, I will debug that later. For the moment I just need the OS functioning enough to verify various things I need for the compiler.

The compiler projects have taken about 1 year, looks like about 26th October I changed plan and began on my own language: I wasted about 3 months on C++. All that code is still there and I will probably continue the C side eventually. No point at all continuing C++, and C will mainly be for bootstrap reasons as the whole idea is to move away from 1978 C,Unix,C++ lineage.

Right now the language + compiler project meets the OS project. No idea if I can complete within 2008 but progress should HUGELY speed up now as I essentially have most of the really tricky foundation subprojects completed. If I dont complete within 2008 I should make GINORMOUS progress during 2009. Reasonably confident I can complete the language + compiler within 2008 and probably make major initial further progress on the OS.

According to my notes I had it functioning on the uniprocessor system, yet now it doesnt. But I think I also was working on different versions of the OS, possibly the version I was focussing on was different and by coincidence the MMU tables were in the first 1MB. Various things didnt make sense this time round, and my notes were vague which version I had been working on! Also the debug output is now much less, possibly that has led to a synchronisation bug emerging with the tower.

============== 6th August 2008 ===============

I am working now on the compiler optimisation. I am going just for moderately lightweight but effective optimisation. This is quite involved but very interesting.

The code output will be the last subphase of the optimisation,

I am also designing integration of the language, compiler and the OS. This is to allow the OS to utilise language and compiler features and for the language and compiler to utilise OS features.

Even now there are some difficult design problems that I havent resolved.

============== 3rd August 2008 ===============

I am ready now to begin work on the last tricky subproject of the compiler.

AFAICT all other subtopics not yet done arent too bad. There is a lot left to be done but other than the next topic it is all downhill now.

Thus the compiler project is moving now to completion. No idea if I can get the OS done within 2008 but I am sure I can get the compiler completed well within 2008.

Porting the compiler to Linux has hugely speeded up development. I still do the main coding on WinUAE, but debug is entirely done via Linux.

============== 1st August 2008 ===============

After many days of effort I have now got 64 bit Linux installed, both Ubuntu 8.04.1 and Fedora 9.

The cross compiler for my language is now ported to 64 bit Linux.

A bug manifested JUST on 64 bit Linux not on 32 bit. After a lot of debug I found the bug which is in some code from many weeks ago.

It took a lot of work with gdb to locate it, the bug was that I convert a list into an array, by first counting how many items to a variable count. Then I allocate the array. Then I rescan the list to fill the array. HOWEVER I forgot to reinitialise count!

With 64 bit the trashing is much greater as pointers are twice as big.

reinitialising the count resolves the bug, and now C's memory allocation functions fine also. ie the calloc bug mentioned on 24th July seems resolved, and it was the above bug.

64 bit Linux is quite impressive,

my own OS project is something different.

============== 24th July 2008 ===============

The recent bug was so tricky I decided my only chance was to port the build to Linux and debug on Linux.

I hadnt used Linux for a long time. Eventually I got Linux functioning. (it took maybe a day)

I began porting the build to Linux. I began getting the source compiling. And then a huge amount of compile errors: 2 files, unportable.c and unportable.h.

I had put all the AmigaOS dependencies in these 2 files. It was major work to "port" these 2 files. In fact I couldnt port them because they rely heavily on dos.library things which CANNOT be ported. I had to redesign the code to workaround things. The new code in fact was a lot simpler but more primitive.

There were a few minor dependencies outside those 2 files, but those were straightforward to redo.

Eventually I got the code functioning. Using gdb I soon located various bugs that were never detected on AmigaOS, eg it found a bug in some code from some years ago. But one bug was elusive. I concluded that it was a gcc bug with calloc(). When I use my own memory system no bug on Linux. When I use calloc() or malloc() the same code crashes both on AmigaOS and Linux. Both are probably using the same bugged source code for calloc.

When I use calloc but alloc 4 extra bytes each time no crash. That means writing to the last 4 bytes is trashing something else which it shouldnt. According to gdb the bug happens within int_malloc() but there are no debug symbols for int_malloc.

My own memory system checks for memory trashing in the next 4 bytes and further and there isnt any trashing. Thus it looks like calloc() is allocating incorrectly (very rarely), probably an alignment bug at the end of a batch of memory.

I cannot prove it is a bug in the gcc memory system, but it looks so.

Using my own memory system I then have continued without problem, locating various further bugs.

I now write the code on WinUAE as the AmigaOS default topaz fonts are THE best fonts for editing. Linux and Windows fonts are all too big or too small. The topaz fonts are just right.

I then try the code on AmigaOS, deal with any bugs which are detected. And then reboot to Linux and try gdb.

I have completed implementing + debugging one of the most difficult subtopics of the project. Progress now should speed up now.

And the code now all builds and runs from Linux. The output isnt meant for Linux but can be done for Linux. There are NO OS or hardware dependencies at all so far with the compiler. I can make it output to ANY OS or hardware.

The compiler now is fully portable, to port to another OS I just need to port the build script which is straightforward to port as the only host dependencies are gcc and rm. NO GNU dependencies at all (other than gcc). I dont use "make" or "sed" or "awk" etc. (the build depends on delete, compile and link).

============== 18th July 2008 ===============

The code so far is now compiled. Trying this out there is a delayed effect bug which I am trying to locate.

After running the compiler the system crashes if I try certain shell commands. Looks like some memory has got trashed.

============== 15th July 2008 ===============

Currently I am doing a study of the work so far. The compiler work is getting very complicated. The work is going way beyond the planned timeline!

A lot of the really tricky things are now dealt with. But there are still plenty of difficult problems left.

============== 9th July 2008 ===============

I have at last completed implementing the equivalent of sizeof. sizeof is straightforward for most languages, but for my language it is very tricky.

At sizeof there is a major conflict of various language features. Very complicated resolving the conflicts satisfactorily. Most of the conflicts have been resolved via very careful changes to the language interpretation.

ie no change to the language but a changed vantage point. its a bit like taking a home computer and saying: same computer but now its a server instead of a home computer.

The machine is unchanged but we change how we view the machine.

Right at the end I realised there is a dependency on the build C compiler being 64 bit. I will probably change that later so that a 32 bit C compiler can be used. The language itself is fully 64 bit, but there is the problem that the build compiler could be 32 bit: building a 64 bit compiler with a 32 bit compiler!

The sizeof code is written and compiled but not tested.

============== 3rd July 2008 ===============

Not done any further work on the projects until today since the last update, have just restarted work. Was working on some non programming projects!

It took me about an hour just to figure out what I had been doing with the language project when I stopped, as things are very involved.

reading through the work I realised various complicating factors, I have managed to deal with them satisfactorily via very carefully chosen language interpretation rules. ie all the code remains unchanged but there are some rules of the form: you CANNOT do this but you CAN do that!

this relates to things where the binary will be too slow or complicated.

anyway the language project is now restarted,

============== 18th June 2008 ===============

115am: I have completed one component of implementing type sizes.

I probably need to study it a few more times. but I will continue with the other components first.

I always used to think that type sizes were straightforward! For something like C they are. However my language is geared quite differently from C and C++.

The problem isnt the sizes themselves but of precise interpretation. an implementation of anything is absolutely precise. But the thing being implemented usually has an element of vagueness. If you arent careful the vagueness may be impossible to make precise without running into various problems such as space inefficiency, speed inefficiency, complicatedness or more technical problems.

The language isnt just a language but a way of doing things and a mindset. Very difficult to create a new way of doing things and a new mindset!

============== 17th June 2008 ===============

I am now working on the equivalent of sizeof, determining the sizes of types.

For a language like C sizeof isnt too bad, but for my language the problem is trickier because of various features and constraints.

Essentially there are various conflicts between the different features and constraints. In order to resolve these conflicts I have had to make various changes, eg I had a set of 3 options for some type feature. I spent many hours on the most difficult option and couldnt see how to implement it subject to the constraints. The second most difficult option was also very difficult and I eventually found a satisfactory way to implement it. The last option I realised could be expressed as a special case of the second option.

To deal with all this I replaced all 3 options by just the second option, effectively removing the most difficult option. The second option achieves most of the things I wanted from the most difficult option and is nice. The special case option is now changed from an expression to an interpretation which is more lightweight to understand and implement.

I have basically dealt with various points of confusion, and the language is much improved as a result. Most of the improvements are based on disallowing certain things. Deciding what to allow/disallow is quite tricky. It is generally true that you can judge a language not by its features but by what it disallows!

its not what you can do but what you cannot do! structured programming is all about "you can no longer do this". because people have found that the extra liberty leads to too many problems.

sizeof for AmigaOS C is actually different from sizeof for gcc. C doesnt actually specify sizes fully. This makes it problematic to use gcc for AmigaOS. Had C specified sizes there wouldnt be this problem!

Anyway I am still working on type sizes.

============== 13th June 2008 ===============

I am gradually progressing with the compiler. I noticed some logical problem with the implementation so far, but after maybe an hour of careful study I realised there wasnt a problem! The code was so complicated and circular that it took a lot of work just to determine whether the problem was dealt with.

There are a lot of different interdependent problems now, and the work tends to be on groups of problems. I am moving towards completion of the current group of problems that I selected to work on now.

I have completed the really tricky problems of the group, the remaining problems of this group are downhill!

Most of the difficult work is the new features of the language.

The language will be great to use, but it is hugely difficult to implement.

An expression such as f( x + g( y ) ) is easy to use, but it is months of work to implement! One of course implements that in generality, ie when you implement that you also implement h( i( a, b, c ) / i( u, v ) ) where eg i has varargs etc.

And I DONT have that FULLY implemented, but have some of the most difficult parts of the problem implemented. eg to implement that you need to implement the type system which itself is major work otherwise you wont have type checking!

In a way that wont be fully implemented until the entire project is complete!

(eg to implement the call you also need to implement the called function f and to implement that you need to implement more or less everything.)

============== 11th June 2008 ===============

I have made decisions now on register usage for compiling functions.

The decisions are in fact quite involved and designed around the constraints of x86. There isnt any right or wrong decision, and the decisions I have made are based on my own opinions about register usage.

With this type of problem all you can go by are opinions,

Its a bit like asking whats the best house you can make using 100000 quid?

For one person the best house could be the most efficient house to maintain, for another person it could be the most functional house: do you want an expensive to maintain functional house? A smaller house means you could use higher quality materials eg smaller but higher quality windows! You could also go for the house which is quickest to build. There isnt a right or wrong answer, but in practice you make a compromise decision: reasonably efficient reasonably functional.

What can I reveal? well I will reveal 2 things: scratch registers will NEVER be used for function args and that a and d are 2 of the scratch registers. But the full set of decisions is quite complicated and took about a day of study to arrive at.

scratch registers not used for function args is entirely based on opinion, and is part of a bigger plan. Perhaps it is ok to have scratch registers as function args with a different better plan! The plan is based around a series of unprovable but reasonable assumptions.

(I define a scratch register to be one which can be trashed by function calls, advantage: time isnt wasted on save-restore, disadvantage: they get trashed by calls! )

what is certain is that this decision is nicer to work with, but then using stack args only is even nicer to work with but slow: and it frees up lots of registers.

Its basically an exclusivity principle, exclusivity seems to be good, except that x86 doesnt have enough registers for clean exclusivity.

The plan I have looks like it will make good use of x86's registers in practical situations.

68k-AmigaOS in fact allowed scratch registers as function args, but that was done by optimising per function the arg registers: advantage: faster, disadvantage: source and binary and compiler become circularly dependent! use a different compiler and a different choice will be made.

It is superoptimal in terms of speed but I think only practical for hand coded asm. And complicates compilers as you have to communicate the choices for each function, namely the AmigaOS fd files.

Basically hand optimised asm will always outdo compilers as human thought is without boundary whereas a compiler will always be oblivious of things obvious to most programmers.

============== 9th June 2008 ===============

I am working on function calls, one problem is deciding on register usage.

But before the compiler will implement the call it needs to first check that the arguments agree with the prototype. That is a problem of correctness, with a compiler one always has to check that things are correct before processing the things. And there are many levels of correctness. Things correct at one level can have errors at a higher level.

Anyway I have been working on checking the correctness of call args today, (in total it is several weeks of work) I think I have that under control now (ie implemented). There are many complicating factors including various things specific to this language. eg I have something comparable to C++'s template functions that makes correctness much more difficult.

Now that is out of the way I can continue working towards the register usage conventions. For x86 I think registers a and d have to be scratch as they are the destination registers for all multiplication and division. a and d are exclusive at the hardware level. x86 multiplies the value in a by the operand and stores the answer at double precision in d and a, the upper bits in d. eg for 32 bit multiply the upper 32 bits of the 64 bit answer are in edx.

Thats a decision! a and d will be scratch, but should they be used for function args? I have to study that further.

============== 5th June 2008 ===============

I have some of function call register usage implemented mentioned yesterday. But many things not done yet.

I am working on various other things as well currently, and am working on other register usage problems. I am working towards a modest amount of optimisation, you can spend 10 years on compiler optimisation as it is a very deep subject.

That is why I only intend to implement a modest amount of optimisation, the register usage problems I am looking at relate to optimisation. The general idea is that if the code does things just with registers it should be faster. However it is a black art as to how to do that, the register usage of function calls mentioned yesterday is one part of the problem. The problem is you could soon run out of registers and also non scratch registers need to be saved at the start + restored at the end.

My compiler is for 64 bit x86, but with PPC the problem is much nicer as they have so many registers, approx 32. Example not necessarily good decision with PPC: have eg 5 exclusively scratch registers, have 8 registers exclusively for function args, that still leaves you with about 19 registers.

that makes the compiler much cleaner. but again you have to decide how many exclusive registers, eg you could have 4 or 6 or other scratch registers,

But with x86 or 68k because there are few registers you cannot really use exclusive registers. You could have one or two exclusive registers eg 68k-AmigaOS uses a6 exclusively for library bases, which is a system level register optimisation. Some 68k-AmigaOS compilers use IIRC a4 as a base register for 16 bit offset addressing.

should you use exclusive registers? how many?

these are difficult questions and there is no right or wrong answer: it depends on the problem.

68k-AmigaOS went for a0,a1,d0,d1 as scratch registers but not exclusively scratch eg d0 could be an argument (eg SetPointer()), a scratch register and the return value of a function.

that is non exclusivity, allowing multiple uses of the same register. note that while d0 is an argument you cannot use it as a scratch arg, you would need to move the arg to another register to free d0 up for scratch use. That is where the implementation starts to get complicated. with PPC you could orthogonalise the uses.

its like everything else, when you have plenty of space you can be more careless eg disk space, ram space, register space.

============== 4th June 2008 ===============

At the moment I am working on function call register usage:

eg scratch registers.

how many scratch registers?

should scratch registers be used as args?

which args should be which registers?

how to deal with varargs?

it isnt clear what decisions to make. something like PPC has a huge amount of registers which makes it an easier problem but x86 has much fewer registers which makes it much more difficult.

this is an implementation problem, it isnt part of the language definition.

============== 25th May 2008 ===============

I think I have completed the semantic processing of function args, this is much trickier than say for C.

Currently I am working on the semantic processing of literal integer consts eg 12345, these are more problematic than you might think...

eg C regards these as ints, but are they signed or unsigned ints?

consider eg 0xffffffff, as a 32 bit int that is -1, but as a 64 bit int that is +(2^32-1), a different number with a different sign!

Another problem is for flags, if you have some 64 bit flags and 32 bit ints then (1 < < 31) could be wrongly interpreted as 0xffffffff80000000 via sign extension.

There are other problems as well, with C you can get around some problems via eg 0xffffffffL, but what if you use a C compiler where long = 64 bits?

I am trying to find my own way of dealing with these and other problems.

============== 24th May 2008 ===============

I have been designing and implementing function arg features such as varargs and optional args.

This has proven to be quite a difficult problem, but I think I have a good design now. It is a different paradigm from the mechanisms used by C and Modula.

The design constraints of my language are quite different from C which makes the problem quite different from C. C's function args are more constrained which makes the problem much more difficult for C.

That is why C's varargs are quite tricky, as their mechanism has to work around the language constraints.

============== 24th April 2008 ===============

The changes to the type system are now implemented and functioning.

It needs more intense testing, sometime I will try and create a test suite and script.

============== 23th April 2008 ===============

Working on the semantic processing I gradually realised that one set of ideas of the type system had a subtle deficiency. It is a meta level design problem.

I have now got a replacement design which does what I want, and am now implementing the replacement idea.

The replacement is in fact a lot simpler and completely different from the original idea.

============== 18th April 2008 ===============

The further problems mentioned yesterday with the type assignability I think are now resolved.

It was a very complicated recursive problem to do with an unusual feature of the type system.

The difficult subphase mentioned on the 16th I think now is complete.

I find that the code for dealing with types is very difficult to get right, but once you get it right then its ok.

The work now is to complete the code for type assignability,

Right now things have been progressing very slowly, this is because of very difficult theoretical phenomena.

Actually using the language will be straightforward with some fantastic features. But implementing that is near impossible.

============== 17th April 2008 ===============

Thinking about the type assignability I realised a problem with the current code. Some complicated assignability was in fact unassignable.

I think I have that problem resolved now.

Meanwhile I have found further problems which I need to study.

============== 16th April 2008 ===============

I have got the most difficult subphase of type assignability coded. Type assignability for my language is a very complicated problem. Further work before I can try the code out.

The system has some features which are VERY difficult to implement,

I knew right from the start of the project that type equality and assignability would be problematic.

============== 11th April 2008 ===============

I am continuing now on the central semantics of the compiler, which is the semantics of code generation. This work is still at the early stages.

I earlier completed the initial semantics of local variables, labels, gotos, and some language specific things.

an example of such semantics is that you can only goto a label which is defined AND you can only goto outwards eg using C notation:

{
goto x ;
   {
   x:
   ...
   }
}

:this is a semantic error as the label isnt visible to the goto. But gcc in fact accepts this, my language rejects this. Thus C uses a different definition of visibility. My language is more structured than C. I cannot see any circumstances where you would need such a goto. even C wont allow you to jump to a label in a DIFFERENT function!

void f(void)
{
goto x ;
}

void g(void)
{
x: ;
}

345pm: I reached the point where I was to implement some feature, but have decided there is a different feature which probably is a better way to do things. I have just completed implementing that alternative which is something I have been thinking about for some months.

I could implement both features but in the interests of efficiency I think I wont implement the first feature. Both features address the same problem but the first feature is a more complex constraint on the language definition.

if I discard the first feature it will free up the language definition and simplify the implementation. The latter feature is already implemented and running correctly.

11pm: semantic processing continues, very complicated problems, but interesting also. I am starting working on the code for type assignability. This is the code to determine if 2 types are assignable eg with C:


int x ; char y ;

x = y ;

with the above int is assignable to char. If the user does an assignment the compiler has to determine if it is valid. Type assignability is another central component.

============== 9th April 2008 ===============

the type equality subproject is "complete" for now. The main functions and features are all functioning correctly. I am sure there are some further bugs and problems but I have run out of motivation to test this any further! I havent tested all features as that is very time consuming but have tested the really tricky ones and resolved various bugs.

Now I will continue with the other semantic processing, all the work now is tricky but I think the language + compiler project is 75% complete.

I will deal with any further type equality problems as and when they arise, and have a further testing session later on in time.

I think I need to create a compiler testing script which tries out different examples to make sure everything is functioning correctly. Earlier in the project there were relatively few things to test but now there are an increasing amount of things to test. I will work on such a script later.

============== 8th April 2008 ===============

Some major features of the type equality code are now functioning correctly and comparing tricky type constructs correctly.

Not all the major features are functioning yet, I am trying to resolve a bug currently with one major feature. With this bug the code freezes up in an infinite loop with some example input. It should only loop a few times but instead loops forever.

Anyway progress is very good,

============== 7th April 2008 ===============

type equality is now implemented and compiled but not debugged.

There is a bug with the first example I tried! Looks like there will be some work to debug the code.

7pm: that bug is now resolved after quite a bit of work. The input example was fairly straightforward, I am sure there will be further bugs.

I am taking a break now before continuing to test the program.

============== 5th April 2008 ===============

I am starting to make good progress with implementing type equality.

But there is a lot of further work to do.

If you email me I can send you a binary of the cross compiler once type equality is implemented. The binary I will send will run on 68k-AmigaOS. That way you can prove to yourself that this project is for real.

============== 3rd April 2008 ===============

The work continues implementing checking for equality of types.

This is one of the most difficult problems I have looked at, its because of the very general nature of types. The problems depend on the definition of the type system, change the definition a bit and the problems also change.

The problems are very high level and very specific.

The main problem is that if you try to test for equality of recursive types that there are multiple dangers of infinite recursion. The danger is that the code never completes. Checking for the type equality is fine, its just that the code is recursive and it is difficult to guarantee that the checking ends.

Its a bit like the following code which never completes:

int factorial( int x ){ return( factorial( x + 1 ) / (x + 1) ) ; }

:this code is correct but never completes!

Ordinary recursive types which you find in C arent so bad eg:

struct a { struct b *b ; struct c *c ; } ;
struct b { struct a *a ; struct c *c ; } ;
struct c { struct a *a ; struct b *b ; } ;
struct d { struct a *a ; struct b *b ; } ;

Proving correctness I completed earlier, the next problem is proving equality. gcc regards c and d as different, whereas old C regarded them as the same. gcc's definition is better as old C's definition is more likely to lead to bugs.

The major problems begin when you go beyond the C scheme...

You have to wait to see how I do that, but you can see an example of going beyond C if you look at C++'s templates. I dont implement templates but I implement my own answer to C++'s templates. My scheme isnt templates at all, but is comparable.

============== 30th March 2008 ===============

I was away on holiday for a bit. On returning I had too many real world things to do but now I am working on the language again.

Right now I am implementing checking type equality. This is necessary eg to determine if a function prototype agrees with the function. Even with type correctness implemented this is very tricky to do for my language. The trickiest thing is recursive types where a set of types are defined in terms of each other.

I am progressing now with this, the implementation is very involved.

============== 22nd Feb 2008 ===============

I am progressing now towards code generation of the compiler. There is a lot of work before any code gets generated.

I have already completed implementing the initial semantic processing of local variables and labels. The compiler now determines correctness of local variables and labels. I have also done some initial processing of some new language features.

I am now working on the initial steps of processing language statements.

============== 19th Feb 2008 ===============

I have the first version of a major component of the language type system implemented and functioning correctly. I have been working towards this since mid November 2007. This is the most difficult part of the type system.

Progress is starting to accelerate. There have been virtually no updates as the work has been so severe.

This work contains a huge number of innovations, a totally different emphasis from C. But C is the yardstick to judge the system by.

There are a lot of things yet to implement. But it has taken me since mid November to get something that functions.

The first attempt to run the code had plenty of bugs but I have gradually been resolving them.

There should be more progress updates now as progress should speed up.

============== 2nd Feb 2008 ===============

I am starting to make more progress now, recently I made nearly no progress with the language.

============== 17th Jan 2008 ===============

At the moment I am working on some very difficult design decisions about the language semantics before I can proceed with the type checking code. This work isnt coding but making decisions, the language will do this the language wont do that. The best decisions tend to be about what the language wont do. But such decisions need to have a good basis.

What I like about C is that there are plenty of things C doesnt do, yet you can do more or less anything with C. For instance old C didnt have const, yet they implemented Unix and C compilers with old C!

(My own language also doesnt have const)

Clever language features are often burdensome.

============== 13th Jan 2008 ===============

I am continuing implementing the type checking. At the moment I am going through the code so far. A lot left to do.

============== 10th Jan 2008 ===============

I have just completed a holiday from my computing projects. Have just restarted the work, a lot to do. I have spent a lot of time deciding on one language feature. It is something which C++ deals with but not C. After a lot of thinking I have decided to exclude the feature from my language. The initial phases of the feature have already been implemented but I have decided to discontinue the feature!

feature exclusion is one of the most important parts of language design, the power of the modula languages is the features they exclude. The features are excluded because they create more problems than they resolve, although it can be very difficult to decide on this.

============== 3rd December 2007 ===============

continuing working on the type checking. Very complicated,

============== 18th November 2007 ===============

working on some very difficult code with the type system of the language.

============== 16th November 2007 ===============

I am continuing implementing the type checking,

============== 15th November 2007 ===============

I have got plenty of coding done and the progress is gradual on the second phase of the compiler for my own language.

I have completed the very low level semantic processing, and am now looking at higher level semantics.

Right now I am implementing the type checking. The design of the language removes a lot of the work that some languages require without any reduction in power.

============== 11th November 2007 ===============

I am gradually moving to the trickier parts of the semantic processing. There are conflicts between various of the design constraints, eg there are conflicts between control, usability and readability. I have had to do some further work with the language definition.

If the language has too much control it also becomes more unusable. Also you could increase usability but at the cost of readability: where its not a problem writing code but tricky to read said written code. If you work on readability you can lose control.

I wanted to increase some of the control, it took a lot of work to avoid unusability and unreadability. Its like sometimes you can buy something which has lots of features but is totally unusable,

I am not attempting to outdo other languages but to create a language which is good in its own way. Relative to other languages there are more of some things and less of others. I think good design is about recurring themes. That is economy of definition. At the same time I dont follow any absolute design principle. That is why I talk of design conflicts, in the real world quality often conflicts with price. If you take any idea too far you can run into opposing problems.

I am also looking at the compiler from the POV of my OS design, creating features which go with the OS.

============== 9th November 2007 ===============

Very busy now implementing the semantics of the language.

============== 8th November 2007 ===============

I am making gradual progress with the second phase of my language. The coding at the moment is the first subphase which is the low level initialisation of the phase. The second subphase is the semantic processing of source files, eg type checking and visibility control.

830pm: the beginnings of the second phase are in place. Trying this out on an example source file in the language it gave an unexpected error message. I found that was something I forgot in the first phase. On correcting that it found a number of bugs in the example.

I have the very beginnings of some semantic processing begun. Languages in general tend to be very convoluted to implement.

============== 7th November 2007 ===============

I have decided to begin now the second phase of the language project. Some obscure things with the transition phase are undecided. But it doesnt matter too much as long as I work just on the decided things.

The decided things are enough for developing my OS with,

930pm: I have started now on the second phase. The initial work is gruntwork coding. The earlier compiler subprojects deal with various things which will speed up development but it takes some work to figure out how to use them.

============== 6th November 2007 ===============

The transition phase of the language project is complete. (transition phase between the first and second phases). I have just begun testing it. This is the foundation of the compiler for the language.

I will then begin the next phase of the language which is a lot of work.

This project is very highly meta, there are many planes of abstraction some of which are severely complicated.

The work so far is CPU independent I can use this for any CPU at all eg 68k, PPC or x86. But I will be looking just at x86.

630pm: I have found a few bugs so far, things I forgot. I am working on one at the moment.

9pm: I am making a few changes as well to the language definition.

============== 4th November 2007 ===============

I have completed the first phase of my language. There is now gruntwork coding before I have some code I can run. This is just an initial phase of the compiler. There are many phases of work before the compiler will be ready to use.

The speed of development is because of the compiler projects I was working on many months ago!

Its like if you design a lamp around a standard light bulb and standard electric supply you dont need to waste time designing a light bulb. Many different lamps can be done according to those constraints.

But you can go one step further by designing a NEW light bulb for use in many different NEW lamps. If you design the bulb well then the many lamps compatible with it may outdo the lamps compatible with the standard bulbs of today.

So here you DO waste time designing a light bulb, but that perhaps makes lamp design easier so you save time each time you design a new lamp for this new bulb. The idea is to not just design a light bulb for a future lamp but to design it for many different future lamps. And if you design the bulb not just to make the lamps more usable but to make the lamps more implemable then future lamp design will be much quicker.

1am: The work now is a transition phase, this is mainly complicated connecting up. I have in fact just completed the low level part of this, next there is higher level gruntwork.

When this transition phase is complete then things start to get very tricky.

============== 3rd November 2007 ===============

Almost completed this current phase. Even by removing a lot of unnecessary things the complexity is almost out of control.

If you try to make a language too powerful it starts to become unusably complicated. The reason C and AmigaOS are so popular is they are very usable. I think usability is very important.

At the same time the system needs to be practical. C and AmigaOS both are very practical. I think the reason C and AmigaOS are so good is they were both created under time pressure. AmigaOS involved huge amounts of hardware and OS innovation. When you are under time pressure you tend to make better decisions.

It is very difficult to make decisions between usability, practicality and other constraints. I have made a lot of language innovations and have resolved many problems that I have encountered when programming. Things which until now I couldnt see a way to resolve. You will get to see these innovations eventually.

One of the tricky things with languages is ambiguities, you try a new feature but find it conflicts with an existing feature. It can be very difficult or impossible to get around some ambiguities. An example ambiguity with C is:

x = y+++z ; 

That can either be x = y + (++z) ; OR x = (y++)+z ; OR x = y + ( + ( +z ) ) ; Which is it? The answer is that C regards consecutive +'s as being ++, so it is the 2nd interpretation. However if you put an extra space:

x = y+ ++z ;

then it is x = y + (++z) ;

And with 2 spaces:

x = y + + + z ;

it is now x = y + ( + ( +z) ) ;

So spaces in C can change the meaning.

============== 31th October 2007 ===============

Trying to complete off this first phase AFA reasonable. It involves very difficult design decisions.

The language is highly notational like C, but using completely different paradigms. Each potential feature has advantages and disadvantages, I think the most important thing is to achieve design harmony.

I need to be really sure of the decisions, because eventually I wont be able to change them. There are still things I am not sure about.

I am designing-in the language functionality my OS requires, I am also resolving many problems that I have observed over the years.

============== 29th October 2007 ===============

Although various things arent decided yet, I am almost ready to begin the next phase of my language. The trickiest things I will return to later.

I am making full use of the compiler projects I was working on some months ago, so this can be regarded as a direct continuation of those. When I began on C++ I thought that would be a direct continuation, nope!

I am doing a lot of things differently from other languages, some things better some things worse.

Why should some things be done worse?

design efficiency, if the advantages of a feature are marginal then I may omit the feature. Especially if the feature complicates use and or implementation.

eg for the moment at least there is no "const" attribute, that is because it complicates use too much.

Basically it is a 100 times less work to do const optimisations in software than to utilise the "const" attribute. If you know something will be const then cache the value to a local variable, but only do this in the few places you need speed.

Also I dont want too many features so the language is easy to learn.

If you look at say ANSI C they remove features in order to keep use and implementation under control. eg:

int f( struct a { int b ; } *c ) ;   // (A)

is disallowed in ANSI C, you arent allowed to define types in function args.

Instead you just do:


struct a { int b ; } ;

int f( struct a * c ) ;

// (B)

Although (A) looks clever, if you look at (B) its a LOT clearer. Implementing (A) isnt a problem but it just complicates the compiler because for each argument of each function there are many options:

the arg type may or may not be defined. eg int f( struct b * c ) ;

the arg type may or may not be named. eg int f(struct {int b ; } * c );

there may or may not be an argname. eg int f(int);

there may or may not be an arg type eg int f(x);

If each compiler feature has many options it will take 4 x as long to learn the language and 4 x as long to implement it. So instead of taking 3 months it takes 1 year! instead of taking 1 week to learn it takes 1 month.

The above is an ANSI C example, which I agree with. ANSI C and ANSI C++ disallow different things! you can write a program accepted by GNU C and rejected by GNU C++ and vice versa. eg ANSI C++ disallows int f(x); From this it is clear that language design is subjective.

Anyway I am making my own subjective decisions.

I am creating a lot of new ideas as well, in fact some new ideas have already been removed for not being good enough! Some ideas are impossible to decide on,

I like old C, that was an authentic language. I think when Dennis Ritchie invented old C it wasnt meant for general use but was just a practical idea for implementing Unix. My situation is similar, I need to manufacture a language in order to implement my OS. I mustnt spend too long on it. Old C and AmigaOS are similar phenomena. Both were practical methodologies with tight time constraints.

My own time constraint is to try and get most of my projects completed within 2008. That is why I switched from C++ to my own language as C++ is too time costly. It is like in an exam, you are answering the questions but eventually there is just 10 minutes left on the clock and you realise you have to cut across a lot of things. Just a sentence where you want a paragraph etc.

The original plan was to use C, and to implement C++, that plan was succeeding but I was starting to see how long it would take: TOO LONG. And I could see that going via my own language would drastically improve the timeline. BINGO!

I think in computing the people who get to the top are the people with political abilities. I want to change that and get to the top via computing abilities. I want to succeed by creating something which is good without cheating my way to the top.

============== 27th October 2007 ===============

I am making rapid progress working on my own language,

I will complete off the C++ work so far sometime soon, I decided to go directly to working on my own language.

When the current C++ work is complete I will shelve that project and will continue the C subset at some point. It is very tempting to work on C++ but looking at the future timeline it will just delay my OS project too much. So far I have had 2 projects which I realised were futile, one was ELF and the other was Java. Both of which CAN be continued but I see no point. ELF is only necessary if you go via gcc, so once I decided to do my own C compiler ELF became unnecessary. Java was an error, I thought I was implementing Javascript, when someone pointed out Java != Javascript, I quit that project immediately, also I really dont like the design of Java.

I am resolving all the endless design problems of C and C++ in my own language. In a few seconds I can avoid a week of coding! The entire language will have an orthogonal syntax.

That is the modern approach to languages. An orthogonal syntax is one where you can process the entire source file without interpreting any symbols. You then only interpret symbols in later phases. To see that C's syntax isnt orthogonal:

x = (y)&(z) ;

Is the & a binary bitwise & OR is it the unary address operator? We can only know by interpreting the symbol y, if y is a type then the & is the address operator, otherwise & is binary bitwise and. So we cannot know just from the syntax, we need to understand the program to know.

The problem with a nonorthogonal syntax is it is very complicated to both interpret AND process the syntax. I achieved that via C-, which you can even try out if you email me. C- essentially orthogonalises the syntax by figuring its way around such nonorthogonalities. If you look at the project updates you will see I have been working towards C and C++ since August 2007, ie 3 months. ALL THAT IS UNNECESSARY FOR A WELL DESIGNED LANGUAGE! eg its all unnecessary AFAIK for the Modula languages, and I think unnecessary for Basic. (I dont know that for sure but think it is)

Now that work towards C and C++ wasnt a waste of time as I will continue that work towards C and COULD implement C++ also anyway! But I can see that it is too inefficient to do that now.

More importantly that work towards C++ has given me very deep insights into the flaws of C++'s design which I will use to design my own language.

The biggest design flaw of all is the nonorthogonality of the syntax, but there are plenty of other flaws. Instead of complaining about them better to implement my criticisms.

Historically speaking I think the idea of orthogonal syntax was AFTER C. C was such a powerful language that people were able to write cleverer programs, they discovered ideas better than those used by C. eg Modula type of languages are usually written in C!

One of the things I will be doing is removing features, eg C++ allows variables to be declared anywhere eg:

void f(void)
{
int x = 1 ;

x++ ;

int y = 2 ;
}

is rejected by GNU C but accepted by GNU C++. The problem with this idea is it makes it more difficult to see whether all allocated things are freed. If all variables are at the start of a compound construct then you only need to look there:

{
// variable declarations

// code

// variable freeing.
}

Basically this is more structured than what C++ allows, it is much easier to check for correctness. Possibly C++ does this to deal with const inits which cannot be done till later. But I am thinking of doing away with the const declaration as it slows down development too much: the disadvantages outweigh the advantages. const certainly has advantages but those arent such a big deal now that CPUs have speculative execution. In the few parts of a program where speed matters you should optimise things in software.

A lot of C++ is workarounds to deal with consts, if you remove the const idea then you dont need those workarounds. Seems worth removing features if it makes life easier for the programmer and compiler. A lot of the brilliance of C was the removal of features of earlier languages. eg C removed call by reference, major C feature: all functions are call by value. But later people have reintroduced that. Call by reference leads to complicated bugs. eg:

void f( int &x, int &y )
{
x++ ;
}

The statement x++ CAN increase y also! eg:

int z ;

f(z,z) ;

Most programmers instinctively assume different names are orthogonal, if you see x and y in a program, then x++ wont affect y. But if you allow reference variables you lose orthogonality of names. So its a good idea to disallow reference variables. The original C idea was that if you want to reference you go via pointers. That way the * operator means other variables could be affected. Put it this way: I can implement C++ without any of these new ideas and without any problems. This may sound strange but I am saying that old C is a better language than C++.

The above ideas arent decisions, the fact is that it is much easier to code with C than C++. The biggest problem with C++ is that the usage is too complicated. There are too many rules, too much thinking to generate code. A subset of C++ is very good, but another subset isnt.

============== 26th October 2007 ===============

I am considering a TOTAL CHANGE OF PLAN! To abandon implementing C++ and instead to create my own answer to C++ instead.

The reason for this is various: I have no intention of using C++ and dont want anyone else to either so its a bit of a waste of time to implement it. I can design and implement something much better in much less time. To understand the HUGE difference this makes ALL the C++ work up to now is unnecessary if I do my own language and compiler.

All the work up to now has been firstly because C uses a pre-processor and secondly because C's syntax is nonorthogonal to its semantics. By working on my own language firstly there will be no preprocessor AND secondly the syntax will be orthogonal. That immediately saves several months of work!

I can then go directly and implement my own OS using my own language. This will save months of time for my project schema.

I think I can complete such at least 4 x as quickly as completing the C++ project. C++ and C are HUGELY inefficient from an implementation POV.

Now I still need to implement C at some point as that is the only way to build my own language above my own OS. Without my own C compiler I can only build my own language from say AmigaOS. But I can implement C later. Its also MUCH MORE INTERESTING to design and implement my own language.

What I am thinking of doing is completing off the current C++ phases of work as that is just a few more hours of work and then working on my own language which should implement lightning fast.

Later on then I will complete the C compiler.

I have begun today working on this and the progress is so rapid that I think I have to do this.

IMO at least 75% of the work of implementing C and C++ is because of the maldesign. There are so many ways to overtake C++ that it would fill a book. A lot of things in C++ are unnecessary, eg C++ allows you to define classes within classes, but I cannot see any use for that. Just move all your classes to the top level. C++ allows namespaces within namespaces, but there is no point to that. You only need top level namespaces. Other languages just use top level namespaces.

C++ is a case study in overgeneralisation and disregard for implementation. In computer science implementation is everything, disregard it at your peril. You can do much more than C++ with much less work than C. Also I want something geared differently from C++, the reason I wont ever use C++ is it is wrongly geared.

Why reimplement C++ when it is much less work to overtake it?

So just as I am attempting to overtake Unix I will now attempt to overtake C++.

To understand the change of direction: I can overtake where I reached with C++ in a matter of DAYS!

You have to remember that C is from around 1978 and C++ is from around 1985, both are dinosaurs.

My plan is to create something which matches all the good points of C and C++ without the bad points.

My earlier compiler projects were all geared to a well designed language, I didnt realise then how badly designed C and C++ were. Anyway now I can use the earlier compiler projects for what they were meant for namely a well designed language.

============== 25th October 2007 ===============

The first version of the current C++ phase is complete and built. However this first version cannot be used yet until I resolve several problems. One problem is that I need to implement some features in C- which this phase depends on. Another problem is that an even earlier phase of the project needs to be modified.

Some features are missing also with this current phase eg I havent dealt with C++'s redefinable operators. I'll probably work on those later.

The phases so far deal with the features I mentioned which possibly I wont implement. The switch construct is done the ordinary way which IMO is the right way. The guys who invented switch had the idea that you test a value and take different actions according to which const it matches.

ie you test a value x and if it is x0 you take action0, if it is x1 you take action1 etc. where x0, x1 are consts known at compile time.

That is what I will implement and that will deal with all situations.

I will implement what I want how I want when I want, its my compiler and I can do whatever I wish with it. I want to use the compiler for constructing my OS. And the OS will only use the C subset of the compiler.

============== 24th October 2007 ===============

I am progressing now with the current early phase of my C++ project. I am implementing my own interpretation of C++ and NOT the official version. My interpretation is how one actually programs with C++ eg it will deal with the code in the tutorial book I learnt C++ from.

Generally speaking it is better to program in C, if you do objects via C tricks you get much better code than using C++. No idea why C++ and Windows are so popular,

An example where my code is different from Stroustrup's definition of C++,

Stroustrup regards a switch statement as having the form:

switch( expression_list )any_statement

This means that he and therefore gcc accept:

switch( 1, 2, 3 );

which is a bit of a pointless liberty. In order to do this he has to regard cases as being normal statements. But they are disallowed outside of switches. So eg gcc rejects:

void f(void)
{
case 10 : ;
}

The only point I can see is that it allows you to bracket off a case:

switch( 1, 2, 3 )
	{
		{
		case 10 : ........
		}
	}

But this leads to problems such as:

switch( 1, 2, 3 )
	{
	while( y )
		{
		if( x > 10 ){ case 30 : t = 10 ; }
		else { case 20 : t = 11 ; }
		}
	}

It just complicates the compiler. In fact to my amazement gcc accepts the above, but that is bad code as you SHOULD NEVER jump into compound statements. You can jump out but not in.

So with my interpretation I only allow the normal form of switch:

switch( 1, 2, 3 )
	{
	case CONST_EXPRESSION1 : case CONST_EXPRESSION2 : ......
		any_statements
	........
	}

default: also allowed instead of a case. Case expressions arent allowed to be bracketed.

That removes a whole wad of work that noone uses!

One problem with bracketing off a case mentioned above is it violates the idea of never jumping INTO a compound construct. I had a look at some open source and found everyone does switch cases the way I do. I think with my compiler I will disallow all jumping into compound constructs. You can jump forwards or backwards but only at the same level or outwards relative to {} constructs. Some C programmers even never use goto's at all as they object to all jumping.

{
a:
   {
   b:
      {
      c: 
      }
   d:
      {
      e:
      }
   f:
   }
g:
   {
   h:
      {
      i:
         {
         j:
         }
      k:
      }
   l:
   }
m:
}

:from c: you can jump to a: b: d: f: g: and m: and not to any other label. Jumping is more complicated with C++ than with C as C++ has automatic freeing of classes when you leave a compound construct: if you jump from c: to a: the code has to free things from the 2 enclosing scopes. I personally NEVER use the automatic classes as I like to control everything at the source level not at the compiler level.

This is all about "structured programming", where you try to only use very regular constructs. C and C++ are very unstructured languages, but by avoiding all the bad stuff you can write structured programs.

The advantage of structured programming is that bugs are less likely and it is easier to write, read and understand code. Also its much easier to write a compiler for a structured language than an unstructured one.

I personally do use goto's but usually I only use gotos forwards and to the same or an outer point in the scoping. Also I dont allow goto's to cross. But sometimes you need crossing gotos eg for a Turing machine:

state1: 

switch( get_symbol() )
	{
	case 1 : action1_1() ; goto state20 ;
	case 2 : action1_2() ; goto state7 ;
	....
	}

state2:
....

You can do that without gotos but its not as fast, just a bit slower. Structured programmers avoid gotos like this:

if( x > 3 )goto xyz ;

........

xyz:

is redone as:

if( x <= 3 )
	{
	.......
	}

xyz:

The problem with that is it increases the nesting of the code, if there are many gotos you end up with excessive nesting eg:

if( x > 3 )goto xyz ;
....
if( y > 4 )goto xyz ;
....
if( z > 5 )goto xyz ;
....
xyz:

becomes:

if( x <= 3 )
	{
	....
	if( y <= 4 )
		{
		.....
		if( z <= 5 )
			{
			........
			}
		}
	}
xyz:

The code is moving rightwards too much and is much more complicated to read than the original goto version. gotos are alright just use them carefully.

I may even change the project to "structured C++" where I only implement the things I use or approve of. That will greatly speed up the development. eg to not implement automatic variables:

xyz  a( 1, 2, 3 ) ; // DISALLOW
xyz *a = new xyz( 1, 2, 3 ) ; // ALLOW

The problem with automatic variables is first they are on the stack which is VERY wasteful of the stack, and second you need invisible deallocation code, so eg if you jump out 2 levels of nesting the jump has to be via 2 lots of deallocations before you reach the label. I can implement such things but it will just delay progress.

Another thing I want to disallow is:

struct { int x, y , t ; } a, *b, c ;
struct z { int u , v ; } w ;

Instead:

struct abc { int x, y, t ; } ;
struct z { int u, v ; } ;

struct abc a, *b, c ;
struct z w ;

Here I am disallowing unnamed types, that is rarely used and greatly complicates the compiler. And to disallow constructs which define and reference simultaneously. One problem with unnamed types is it complicates error messaging AND makes it unfeasible to allocate.

If you look at the above 2 fragments you will see the second one is much clearer to read and understand.

With real code, struct definitions tend to be in header files and variable declarations in source files.

Another thing I want to disallow is:

typedef struct x { int y, z ; } a, *x_t ;

This is even worse as it is doing 3 things: it is defining the struct, then the typedef is referencing the struct to define further types. Instead:

struct x { int y, z ; } ;
typedef struct x a, *x_t ;

This way typedefs only reference pre-existing types. Again this simplifies the compiler implementation.

ANSI C++ itself disallows various things allowed in C, eg C allows

int f( x, y , z ) ;

which noone uses, whereas ANSI C++ disallows that, instead you need:

int f( int x, int y, int z ) ;  // (A)
int g( int, int, int ) ;  // (B)

In fact (A) is better than (B) as you can create the prototype (A) just by cut and paste from the actual function. So maybe I should disallow (B).

Although (B) helps the compiler its NO USE for the programmer as you dont know which argument is what, whereas if you have:

int g( int width, int height, int count ) ;

Then you know exactly what to do. So I think I will disallow (B). If you look at the AmigaOS proto includes you will see ALL args are named.

GNU C++ seems to reject functions defined in functions, so I probably wont implement those! GNU C OTOH appears to accept them. Its not a big deal to implement them but I dont see the point,

These types of things improve code quality and simplify the compiler. I havent decided yet if I wont implement the above, but I dont want to implement them.

============== 22nd October 2007 ===============

I tried implementing C++ the way C++ is described in Stroustrup's book (Bjarne Stroustrup being the guy who invented C++), but that led to endless problems. So now I have reverted to the commonsense approach implementing C++ according to my interpretation of it. This is making better progress.

Although I am implementing C++ I only guarantee to attempt to implement the C subset of C++. I am only attempting to implement C++ as that will avoid duplication of effort.

I find Stroustrup's book is only good if you already mostly know what you are doing.

C++ generally tries to be too clever, eg it has 5 different ways to do typecasts, NONE of the new ways I have ever used! eg const_cast allows typecasts where ONLY the const-ness is altered. The thing is that in a well written program typecasting is rare so you shouldnt waste language definition on rare phenomena otherwise you get an inefficient language definition. In fact I mostly dont use the "const" construct as it slows down development too much. If I use "const" I have to perpetually stop and think: "is this a const?". The main advantage of "const" is it allows the compiler to cache the value of a const in a register. But todays computers are so fast that it doesnt make a lot of difference. Its only a tiny subset of a program that determines speed, in those cases you can optimise for consts in the source by caching the const in a variable in the program.

The other problem is that a non const variable could be locally const, eg window_move(....) changes x,y but doesnt change w,h. And window_resize() changes w,h but not x,y. So eg window_move(....) clearly doesnt change the area of the window.

============== 19th October 2007 ===============

I have completed the initial subphase mentioned yesterday. I have been looking at some of the amendments to C++. I cannot guarantee to implement each and every change they make to the language. When programming with C++ try and avoid things which COULD be interpreted more than one way.

Almost ready to begin the next subphase which is more complicated.

============== 18th October 2007 ===============

I have almost completed an initial subphase of work, this subphase is timeconsuming but straightforward. The subphase contains various problems so the next subphase will be working around those problems.

============== 17th October 2007 ===============

I am busy now working towards C++, there's a lot of gruntwork before there is anything of interest. The C- idea is making the work much better than it had been when I first tried to directly implement C++.

============== 16th October 2007 ===============

I am restarting the C++ compiler project now. Missing functionality in C- will be worked on as needed. For the OS project in fact I just need the C subset of C++ to function.

Anyway C- now patches the ambiguities of C++, all of which relate to the ambiguity between types and variables.

An example ambiguity is something like:

x::y::z < t, u > ::v (w) ;

That could either be a variable declaration of w or a function call with argument w. This particular example isnt too bad as the syntax is the same either way. But the problems are worse for say:

z = (x)&(y) ;

& could either be binary & or it could be address & with (x) a typecast. Those are syntactically different: binary & is binary_and(x,y) whereas unary & with typecast is type_cast( x, address(y)). The bracketing is totally DIFFERENT, the latter fragment has 2 operators the former has just 1. That is just bad design as it means the syntax is DEPENDENT on the semantics. With a well designed language the syntax is INDEPENDENT of the semantics and you can process the syntax first disregarding semantics. If syntax is orthogonal to semantics that makes it MUCH EASIER to implement a compiler. Via C- I can patch the problem, C- figures out which things are types and inserts @ before those. That orthogonalises the syntax, allowing the next phase to be much cleaner.

So if & above were the address operator then C- changes the fragment to: z = (@x)&(y) ; .

============== 14th October 2007 ===============

A first version of C- is now complete. This probably does have bugs and some missing features. AFA C goes anything missing should be straightforward to implement. And AFA C++ goes it should deal with NORMAL programs with missing features straightforward. If you really push C++ to its pedantic limits then there could be problems.

One feature I havent implemented yet is "friend", I will implement that later on. A complicating factor is a friend declaration to a function not yet declared, which scope is it in? C++ does specify how to deal with this. TBH C++ is a bit overanxious eg my C++ compiler will be ENTIRELY written in C where ALL fields are public. Stroustrup gives the example of multiply( matrix , vector ), where for speed multiply needs to access private fields of both matrix and vector. So he makes multiply a friend of matrix and vector.

The real problem is that the object paradigm has limits and multiply() is something BEYOND the limits of the paradigm, so he has to hack the object paradigm to deal with it via the "friend" idea.

Objects are variable centric, multiply() is an API centric problem. C++'s problem is its vantage point is wrong, C++ is an object centric language. Real programming is API centric.

The C++ tutorial book I have doesnt mention "friend" at all so it must be a more obscure feature of C++.

============== 13th October 2007 ===============

The C- "using" bug took some work to resolve but its resolved now. The "using" directive means C++ tries to resolve things relative to several namespaces. On resolution the compiler must still check the remaining namespaces: multiple resolutions mean an ambiguous resolution.

using namespace x1 ;
using namespace x1::x2 ;
using namespace x3::x4 ;

typedef y z ;

:if

x1::y
and
x1::x2::y
are both defined then
y
has an ambiguous resolution. The compiler attempts to resolve y relative to all 3 namespaces and the top level namespace.

The bug was that on finding a resolution the later unsuccesful resolutions were overriding some of the successful resolutions parameters resulting in a failed resolution. I just had to backup all the parameters on success. Finding the bug was the tricky thing, resolving it was just a few lines of code.

Just one tricky thing left to do and I will probably then begin on the C++ compiler.

============== 12th October 2007 ===============

I completed implementing the "using" directive today for C-. Currently there is a bug with this so I am working on that at the moment.

============== 11th October 2007 ===============

The most difficult part of C- template classes are now implemented and built but not fully debugged. Working on a bug at the moment. There are various things not yet implemented. I really dont like C++ at all, its too complicated and the syntax is too inefficient. You can do much more with much less. Templates are a very inefficient and complicated way to do things, a total pain to implement.

============== 8th October 2007 ===============

I think I have the trickiest part of the C- templates implemented. I am not completely sure as templates are very complicated to implement. I need to study very carefully what I have done. Using templates isnt too bad but to implement them is very tricky. Even GNU's C++ has bugs with their templates, thats a sign of bad language design if mainstream compilers have difficulty implementing a language.

630pm: I found some problems I hadnt dealt with. Those are now dealt with. Looking now for any further problems. Later implementing C++ will be very similar so this in a way is good preparation for that.

============== 6th October 2007 ===============

The work on C- templates is getting ever more complicated, nonetheless I am making progress.

============== 5th October 2007 ===============

I am working on C- templates at the moment. This is the most difficult part of the C- subproject. The main use of templates is for recursively variable types: variable types with variable "subtypes", for example:

template < class x, class y > class z
	{
	template < class a, class b > class c 
		{
		a a1 ;
		b b1 ;
		x x1 ;
		y y2 ;
		} ;
	} ;

Here c is a variable "subtype" of the variable "type" z. (I say "subtype" and not subtype as that is probably the wrong word but its the best way to refer to the phenomenon.)

It took many days of effort to find a way to implement such variable types. I am currently implementing the plan I eventually arrived at. All the C- and C++ work runs above 68k-AmigaOS as a cross compiler.

C++ templates allow references to nonexistent things eg in the above fragment you can refer to eg

x::t
which is nonexistent nonsense, this leads to an unresolvable ambiguity that we dont know if x::t is a type or a variable. GNU C++ deals with this problem by ALWAYS assuming
x::t
to be a variable UNLESS it is preceded by the keyword "typename". (something I learnt from GNU C++'s error messages!)

Stroustrup the inventor of C++ doesnt mention typename in his book, so he probably wasnt aware of the problem. Anyway I will use GNU's typename idea to deal with the problem as that seems a reasonable idea.

It does appear to prove that C++ as described by Stroustrup is truly ambiguous.

I dont know why everyone backed C++ as it is such a royal pain to implement.

============== 16th September 2007 ===============

The C subset of C- is complete and the C++ subset is mostly done, just one tricky thing left to implement is templates. C- so far will catch a lot of errors. But some things it wont, and it allows a certain amount of nonsense. Originally I was going to prefix things like

a::b::c
by
class
wherever the construct was a type. But that conflicts with C++ eg the following is wrong in C++:

typedef int a ;

class a z ;

With GNU C++ class and struct can only qualify something which is either a class or struct, eg struct can qualify a class and vice versa.

So instead C- will prefix such ambiguities by @, I think some of the GNU build tools do customisable C code using @ symbols. The @ symbols will be prompts for the C++ compiler to know whether a construct is a type or a variable. It is much work to determine which. An example of what C- will do:

namespace 
	{
	namespace a
		{
		typedef struct b { typedef int u ; int v ; } w ;
		}
	}

a::b::u f( a::w::u x ) ;

will be redone by C- as:

namespace 
	{
	namespace a
		{
		typedef struct b { typedef int u ; int v ; } w ;
		}
	}

@ a::b::u f( @ a::w::u x ) ;

So all that C- does is insert @'s, but that is a complicated thing to do and it has to deal with multiple inheritance and templates. Currently it ignores private and public qualifiers as any errors with those will be detected by the C++ compiler. The inserting of @'s AFAICT removes all the unpleasant ambiguities of C++. I could actually continue C- to be a C++ compiler but it looks better to do C++ as another phase.

============== 12th September 2007 ===============

I have made some progress now towards C-. Where there is a conflict between C and C++ I will go for the C++ idea.

eg C++ doesnt allow the following 2 lines of code:

int f(x,y) ;
int f(x,y){ return( x + y ) ; }

whereas C allows both. With C++ you have to do:

int f(int,int) ;
int f(int x, int y){ return( x + y ) ; }

which is also accepted by C, so I will only allow the second fragment. The latter makes implementing the compiler easier as you dont have to deal with int args as a special case eg there is a further complicating factor for C if you have:

typedef double x, y ;
double f(x,y) ;

Are the arguments of f integer or double?

Probably double, but the problem is that if you are reading some code and there are endless header files you dont have the time to find whether by coincidence x and y are types. Disallowing this means that you ALWAYS know that a prototype argument ALWAYS starts with a type.

C- at the moment does some correctness checking, eg it will deal with:


typedef struct { typedef struct { typedef int x ; } y ; } z, (t), u ;

typedef t::y::x v ;

As you can see, with C++ things are much more complicated than with C.

Notice that C++ structs are almost identical to C++ namespaces.

============== 10th September 2007 ===============

The work towards C- is proving very tricky. I am progressing but very gradually.

============== 5th September 2007 ===============

I want my system to deal with gcc C++ headers, gcc is using really obscure C++ functionality. Admittedly that is just header stuff so my own headers dont need to use such things. it has things like:

struct x : public y { ..... }

And I thought that was only allowed for classes. A strange construct gcc uses is:

extern "C" { .... } 

It does lots of such strange things, here is another one:

__extension__ typedef long long _G_llong ;

What on earth is

__extension__
?

It is all quite infruriating. I dont guarantee to support all such things as I dont want to spend 10 years implementing stuff that NOBODY uses.

============== 4th September 2007 ===============

I managed to resolve the compiler subproject bug. I tried the preprocessor on a demo C++ source file, there was a preprocessor bug with this. It took a lot of work to locate the bug. I am continuing now on the first phase of C-. I have 4 languages to consider: C-preprocessor, C-, C and C++.

The initial version of the first phase of C- located some problems in the AmigaOS includes: some function prototypes use an arg name "template" but that is a keyword of C++.

11pm: On a certain example file the system freezes up after trying the preprocessor, trying to determine the problem.

1am: at last resolved the freeze up, it was to do with resetting the current directory and unlocking directory locks in the wrong order on AmigaOS. The preprocessor changes current directory to the directory of the include it is currently processing.

============== 3rd September 2007 ===============

Trying out the initial version of the first phase of C- on a real example has led to a very deep bug in the compiler subprojects. I can see 2 possible ways of dealing with it, am making a study of those

============== 2nd September 2007 ===============

In order to continue towards C++ I have had to make some major improvements to the earlier compiler subprojects.

I am continuing now with the C- project. Its a very unusual project as it is both superficial and difficult.

I have got an initial version of the first phase of the C- project, I have been trying this out on a real source file and resolving the problems. Right now it has gone 3% through the file before halting on "extern", so I have just corrected that. C- is supposed to understand ALL C and C++ source files.

============== 21st August 2007 ===============

The C++, C and C- work is proving to be very tricky,

I think today was the trickiest day of all my projects so far. I dont even know if the C- idea will work, even if it does C is a very tricky language, for instance C's function types are almost impossible eg:

void (*(f( int (*g)( int, int ) )))(int (*)( int (*)(void) ) ){ return( 0 ) ; }

That's not a void function even though void is written there! (if you compile it with gcc you will see no error)

Although C is nice to code with, implementing it is an adventure into hell.

============== 20th August 2007 ===============

A new ambiguity which I think I cannot deal with:

x(*f)(a);
This could either be a declaration of a variable f, pointer to function with argument type a and return type x. OR it could be a double call, x called with arg *f and the returned function pointer called with a as arg.

So it looks like I have to go for a dialect of C(++) similar to gcc's. My implementation though will be quite different, my plan is to create a low level language C- which takes a preprocessed C file and inserts a marker keyword at all

a::b::c::d
expressions which are types. That will remove all the ambiguities and will greatly simplify the C++ compiler.

330pm: I am working on the above idea now, if

a::b::c::d
is a type it will be redone as
class a::b::c::d
. I think I can implement the C- and C++ as the same program, you run the program twice to compile a preprocessed C file.

515pm: change of plan, I will do C- as a standalone language, it looks too complicating to superpose C++ and C-.

630pm: I am working now on the C- project!

1230am: the C- work continues, this is a gruntwork subproject.

============== 19th August 2007 ===============

Another ambiguity:

x (y) ;

That could either be a variable declaration of variable y of type x, OR it could be a function call, function x with argument y.

The irony is that a well designed system is much less work,

I have had an idea, the way to deal with all these situations is to avoid unnecessary bracketing. So if the compiler sees an ambiguity it will assume the bracketing is necessary:

x (y)
is a call,
x y
is a variable declaration.
(x)&y
is a typecast,
x & y
is a binary op,
(x)(y)
is a call,
(x)y
is a typecast,
sizeof x
is a variable,
sizeof( x )
is a type,

The only possible problem is that an expression could be from a macro, I think in the rare cases where that happens you have to work-around things somehow eg the user supplies the bracketing. Usually macros rebracket all args, but for the rare cases where the above idea conflicts, the caller of the macro can supply the bracketing. I'll wait till I see an actual example.

If the compiler interprets different from what the user meant it will always generate an error.

8pm: So far the above idea looks feasible, it does mean the compiler is not completely compatible with gcc, but it simplifies the implementation. And it must be very rare for there to be any problems. The central ambiguity is that

a::b::c::d
can be either a type or a thing, and that the interpretation as a type completely conflicts with as a thing for several of the ambiguities.

The advantage of the above idea is it is entirely syntactic and local.

145am: one further ambiguity:

x *y ;
can either be a variable declaration or an expression statement. Clearly such an expression statement is meaningless so I think I will deal with this by disallowing such meaningless expression statements. What possible point can there be to a statement "x * y ;"? email me if you can see any point.

As with the earlier idea, any misinterpretation will result in a compiler error, so it cannot lead to problems. If these ideas conflict with another compiler then all you do is conditional compilation via #if etc!

The advantage of these ideas is it creates a clean dialect of C, it basically orthogonalises the phases of compilation. I think gcc's approach must be nonorthogonal with dependencies between the phases.

============== 18th August 2007 ===============

Further to yesterday I have found another C ambiguity:

(x)(y)(z)(t)
that could be a sequence of typecasts OR it could be a sequence of function calls!

ie

x, y, z
could be types or they could be functions.

Each of these ambiguities is extra hassle.

There are endless complicating factors, here is one: in a for loop the first part of the "for" is a list of statements, but can any statement at all be there? eg is the following allowed?:

for(  for( i = 1 ; i < 10 ; i++ ){ printf("hello");}, int j = 0 ;  j < 100 ; switch(j){ case 0 : ;})
	{
	}

Having say

for( i = 1 ; i < 10 ; )i++ ;
within the initialisation of another for loop will create a problem because of the semicolon, not sure if you have to omit the semicolon then eg:
for( for( i = 1 ; i < 10 ; )i++, x = 10 ; ; );

The only way to decide is to see what gcc does, if gcc doesnt allow something then I wont. As mentioned on 11th August gcc tends to disallow tricky scenarios.

Kernighan and Ritchie, and Stroustrup say absolutely nothing about this problem, Stroustrup appears to have just copied K&R and gcc rejects for loops in another for loop's initialisation. In fact K&R refers to the initialisation as an "expression". (K&R first edition page 56 says

for( expr1 ; expr2 ; expr3 )statement
, but expr1 isnt an expression. The book says "grammatically the three components of a for are expressions.")

Well a true expression is a bit pointless, but gcc DOES appear to allow expressions as statements eg "1==1;" is a valid statement with gcc even though it is pointless.

Semantically speaking the first part of a for is statements, the second part is a boolean expression and the last part is reinitialisation statements. Note that for and switch statements arent expressions, perhaps that is the idea. Its an incoherent setup eg it allows

for( 1, 2, 3, 4, 5 ; ; );
which is jibberish.

============== 17th August 2007 ===============

Continuing with the initial C++ work, the work is complicated and slow moving.

One problem is that C has various ambiguities eg:


typedef int y ;

void f(void)
{
int y = 5 ;
int x ;
int z = 10 ;
x = (y)+z ;
x = (y)-z ;
x = (y)&z ;
}

The last 3 statements are all ambiguous as

(y)
could either be a typecast of the 3 unary operators + - and & OR y could be the variable and the + - and & are binary operators. I think if both have meaning then its a typecast, it is just irritating that such an ambiguity exists. That is quite a major ambiguity as its an ambiguity whether the operator is unary or binary.

There is also an ambiguity with sizeof:


typedef struct { int a, b, c ; } x ;

int f(void)
{ 
int x ;
return( sizeof( x ) ) ; 
}

Is that sizeof of a variable or a type? gcc appears to be going via the local variable, but that is nonetheless an ambiguity eg someone could have sizeof(x) in a macro intending the type, but the macro could in fact give the size of a local variable. Ambiguities are ok if they are ambiguities of things of the same category, eg an ambiguity between types, between variables, between functions. But an inter-category ambiguity is incoherent design eg between a type and variable,

============== 15th August 2007 ===============

I have some very initial work towards C++ done. Some very low level gruntwork,

============== 13th August 2007 ===============

130am: the C preprocessor is now preprocessing the example file from the compiler project correctly. There were several bugs. I find the preprocessor quite a painful project. My pre-processor is stricter than gcc eg gcc allows repeated definitions without error eg:

x.h is:

#define X  1

y.c is:

#include "x.h"
#include "x.h"

int x = X ;

gcc will compile that without error! My pre-processor will give an error message of a redefinition, it then uses the new definition and continues.

I think I am now ready to continue with the C++ project.

8pm: I am coding now C++. This is a continuation of the compiler subprojects so far. The C++ compiler will compile the output of the pre-processor project.

============== 11th August 2007 ===============

Trying out the pre-processor I realised a tricky problem:

#define y  100

enum x { y, z, t } ;


void f(void)
{
int y ;


y: ;
}

The pre-processor is meant to be run first, but then the y in the enum will be substituted by 100. The "int y" will become "int 100" and the y: will become 100: If they didnt that would be very tricky to code, fortunately gcc does strictly substitute the y's and then you get compiler errors in each case. I am thankful that gcc substitutes the y strictly in all situations. Looks like each potential problem will be caught by the compiler when it works on the preprocessed code.

The OS project will only use C and asm, and the C++ compiler is being done in C. The reasons for implementing C++ are various, eg if I only implement C it wont be forwardly compatible with C++ so I will have to do yet another project to implement C++. C++ OTOH is backwards compatible with C.

I am not completely enthusiastic about C++!

Some of C++ is very good and some isnt.

============== 10th August 2007 ===============

I am working on the current pre-processor bugs before continuing towards C++.

I have resolved the bug:

#define x() y

where there are no args, there is a further bug that if I call the macro eg:

#define x() y
x()

that there is a bug with this. I will look at that next. There is just one further C++ topic I need to study which is the C++ exceptions on which I need further info in order to implement them.

530pm: all the empty arg bugs are now resolved. The preprocessor code for #define is very complicated. There may be further bugs elsewhere. The pre-processor is now preprocessing gcc's < stdio.h > without error. I want to implement one further feature for the preprocessor and then will continue with C++.

1130pm: the gcc example now preprocesses correctly. I tried next to preprocess one of the compiler project source files, that revealed 2 further bugs. But interestingly it located errors in the Geekgadgets AmigaOS includes, eg exec/io.h has:

#ifndef EXEC_PORTS_H
#include "exec/ports.h"
#endif /* EXEC_PORTS_H */

That is WRONG! as it should be EITHER

#include "ports.h"

OR it should be:

#include 

"somepath" in C represents a filepath RELATIVE to the current file, the above is wrong, there were plenty further such errors!

============== 9th August 2007 ===============

I am gradually getting the 64 bit C++ compiler project started. I have been doing a study of C++ for that and figuring out how to utilise my various compiler subprojects for this.

I will try and implement features such as templates, multiple inheritance, operator overloading etc.

I decided to try out my C pre-processor which I completed last November on the gcc includes via a small test program. It malfunctioned at one point, I have that bug resolved now. I retried it again and there is another bug it doesnt deal with macros with 0 args eg:

#define x()   y

I will try to correct that bug. The preprocessor is meant to be compatible with any C compiler, in particular it should be compatible with gcc.

============== 2nd August 2007 ===============

I am restarting my compiler project. According to the updates I completed the C preprocessor 25th Nov 2006, almost 8 months ago! Just doing a study of where I reached. The build for the C preprocessor is highly convoluted, am just making sense of that.

The project will now be to create a C++ compiler and not a C compiler as originally planned.

============== 1st August 2007 ===============

330pm: I am making further progress now with the bug mentioned yesterday. I now know more precisely what the problem is. It is a coincidence situation where 2 completely different phenomena happen at exactly the same time and the code overlooked this situation. I havent yet located where in the code the problem happens but am progressing. There may well be further bugs.

335pm: it looks like a trivial error: I wrongly had jne where it should have been je! je deals with the coincidence, jne leads to garbage. It has taken more than 2 days of impossible work to locate that!

1130pm: well that one wrong ascii character removed now has both the tower and laptop launching 6 new tasks and then the launching task gets frozen out. The launched tasks are functioning fine, so I now need to debug why the launching task gets frozen out. Anyway it is major progress.

220am: this next bug is proving equally evasive.

430am: found the next bug, about to try that out. In fact above there was an error message but the output of the other tasks obscured that. I eventually found another spelling error!

5am: WHEEEEEHOOOOOOO! The new multiprocessor pre-emptive multitasking system is now fully functioning. In the experiment the launching task and all 11 launched tasks are functioning both on the 1-cpu and the 2-cpu systems. This new system tries to relaunch tasks to the same CPU whenever possible. This is to optimise cache usage, sometimes tasks will be relaunched to a different CPU. This is done via a complex heuristic optimisation. The effect on the experiment is quite dramatic. With the original system each successive echo is on a different CPU (see earlier video) eg: 111[ 0 ] 222[ 1 ] 444[ 0 ] 777[ 1 ] ... the cpu keeps alternating. With the new architecture the output is more unusual eg: 111[ 0 ] 555[ 0 ] 333[ 0 ] 888[ 1 ] 777[ 0 ] 444[ 1 ] 222[ 1 ]...

In this hypothetical output the architecture is preferring to relaunch task 5 to the same CPU as the previous task. The original architecture would have relaunched task 5 to cpu 1. The acid test is whether things run faster, that will have to wait till later. I will try and upload a video of the new architecture sometime.

I should say that currently I have only tested out all tasks being the same priority. Eventually I will try out unequal priorities, so there could be bugs in that.

I think I am ready now to restart the compiler project, the aim now is to implement a C++ compiler which will be 68k-AmigaOS hosted. I will make a 68k-AmigaOS hosted demo of this available, email me if you want to try that. It is too early for betatesting as that would require betatesting the OS itself which is no-can-do at this point.

============== 31st July 2007 ===============

Making very slow progress on the bugs.

315am: After some 2 days of effort I have at last determined one scenario which freezes up the system. Its a needle in a haystack problem, the bugs can be anywhere at all. I have to continue with this tomorrow.

============== 30th July 2007 ===============

I am looking at difficult multiprocessor bugs. At the moment the bug is that the one CPU locks A and then attempts to lock B. The other CPU has B already locked and attempts to lock A. Both wait forever. I thought I had designed the system so this couldnt happen. I know what the problem is but havent yet determined the 2 places in the architecture where the conflict is arising.

Have found another similar bug, so far none are resolved, it is proving very difficult to locate. Just about to try another debug build of the OS.

1015pm: I am studying the code, I have found one bug already. I have just removed that and will see if there is still a problem. If I can get the code to function this week I will be doing well!

============== 29th July 2007 ===============

Almost 3 weeks since the last update! I implemented some major multiprocessor enhancements to the multitasking. I have now split up the current module of the OS into smaller submodules. I also made space improvements: eg removing obsolete code and replacing some macros by functions etc, I managed to reduce the size of the OS by 67K. I am now debugging the new multitasking enhancements. Just about to try out yet another build.

Well, on the uniprocessor 6 of the 12 tasks run without problem and the other 6 dont appear to be running. On the dual core none of the further tasks run. Looks like I have some debug work to do!

750am: after a lot of work on the 2-cpu system I now have one of the further tasks running. A lot yet to do.

============== 9th July 2007 ===============

I am working on some major improvements to the multiprocessor multitasking. So at the moment the code doesnt function but I have a backup of the last functioning version just in case.

============== 8th July 2007 ===============

I spent most of yesterday on one of the trickiest bugs yet. That bug is now resolved.

Anyway the semaphored text is now functioning perfectly, I may upload a video of this. It is a very interesting phenomenon as the tasks AND cpus change after each echo eg 111[ 0 ] 444[ 1 ] 333[ 0 ] 888[ 1 ] ... It goes through all cpus and all tasks before repeating.

This means the output is very similar regardless of the cpu count.

Here is a video of the text semaphores. This time I set the build to use white text on a blue background so it looks like an Amiga 500! I can toggle off and on the text by pressing t on the keyboard, this is done by toggling the echo string function with a no-op function. That isnt in the video as it could be misunderstood as a malfunction!

I am working now on some major enhancements to the multitasking, this is a subproject.

============== 7th July 2007 ===============

Trying to get the improved semaphored text to function. Finding very tricky bugs. The code isnt yet properly functioning, so the multitasking subproject isnt complete yet.

============== 6th July 2007 ===============

A specific build of the system was freezing up on the tower each time. I eventually found the bug which was a garbage pointer. :it was a precise coincidence bug, the bug was only rarely invoked. But it happened each time at exactly the same point JUST on the tower.

I am now doing gruntwork eg most of today was making the text echoing object oriented (in asm!). The reason is that once I implement graphics I will need totally different text rendering via the same code. It was working on this that the bug above happened.

Next I am going to redo the text echo via proper semaphores. As you can see in yesterday's video all tasks access the same space for text, so only 1 can be allowed access at a time. With 2 cpus the system would crash if both accessed the screen at EXACTLY the same time. So when 1 task is writing, other tasks need to be held off with semaphores.

Much later on there will be windowing and tasks will have their own private windows to echo text in.

Looks like I cannot use proper semaphores with my text as that interferes with the debug system. Text in general can be semaphored but not this particular low level text. But I can use an improved version of the text. I am running into some really strange problems eg testing out some things at first everything looked alright but eventually there were just 2 tasks left. All the others disappeared! I eventually found a bug in the system where tasks were vanishing. Any task which reached that part of the system vanished! That problem is resolved. I now keep verifying that all 12 tasks are there. One of the "features" of the architecture is proving to be a bug. So I have removed that feature. I spent a lot of time on that feature and didnt foresee the problem. I am looking at an inefficiency problem at the moment.

============== 5th July 2007 ===============

The 64 bit multiprocessor pre-emptive multitasking seems to be functioning fine now. I havent been able to get further malfunction. There could still be bugs so I will continue to study the code.

I think I will upload a video of this in action.

here it is. This shows the tasks once all are running. Task 0 echoes 000[ m ] repeatedly where m is the cpu number, task 1 echoes 111[ m ], ..., task 10 echoes 101010[ m ], and the launching task echoes zzz[ m ].

So if you see 111[ 0 ] 444[ 1 ] 111[ 0 ] 444[ 1 ] ... that means task 1 on cpu 0 wrote 111[ 0 ] then task 4 on cpu 1 wrote 444[ 1 ] etc. The cpu numbering is an OS numbering of the cpus. 0, 1, 2, ...

From this you can see that there are 2 cpus, [ 0 ] and [ 1 ] and also that the interleaving is PERFECT. the 2 cpus perfectly alternate echoing. Each time I press a key the next screen of text occurs. Because of the way it is written task changes happen very soon after the key is pressed so each screenful tends to be a totally different pair of tasks. Later on I keep the key permanently depressed. This text echo isnt ideal as it doesnt use proper semaphores but uses hacked inefficient semaphores. Later I will redo it with proper semaphores. All the tasks here are the same priority,

If I run this on the tower which is 1-cpu you will see just eg 111[ 0 ] 111[ 0 ] 111[ 0 ] 111[ 0 ], so you can tell how many cpus there are by running this demo. Here is the demo of the 1-cpu tower version.

============== 4th July 2007 ===============

The debugging continues, I have just found a fairly major bug where I forgot to check for some bit! I am just about to try that out now.

715pm: several bugs later and it still isnt functioning. I have just now managed to locate one part of the problem.

840pm: made a lot of progress on the bug, not yet resolved.

10pm: looks like the central problem was a typo! The tower is now correctly functioning, will try it on the laptop next.

1115pm: well, it is almost functioning now, still some further bugs. On the 2-cpu laptop several tasks get launched and everything looks correct but at some point only 1 ends up running. The other tasks are frozen out, I'll continue working on this tomorrow.

340am: I put some debug code but that has affected the behaviour as now each time on the laptop the 12 tasks pre-empt perfectly. So there is a bug somewhere but it is a very precise bug.

With uniprocessor multitasking if task 1 echoes 111 111 111 ... and task 2 echoes 222 222 222 then you get:

screen(s) of 111 111 111 and then screens of 222 222 222 ...

However with multiprocessor multitasking you get interleaved output: 111 222 111 222 111 222 as both cpus are SIMULTANEOUS. The interleaving when I tried it out was PERFECT, with 12 tasks and 2 cpus it is even more fascinating. I'll upload a video once I have debugged things further.

============== 3rd July 2007 ===============

I am continuing with the work towards pre-emptive multiprocessor multitasking. This is a very complicated problem as there are a lot of potential coincidence bugs. There is a problem of a system which functions most of the time but eventually malfunctions.

615pm: I have just tried an attempt at the work, it malfunctioned. Trying to find what the problem is.

930pm: I am making some progress with the bug. The code malfunctions differently on the uniprocessor tower from the dual processor laptop. Its going to take some time. Very complicated problem as there are multiple tasks on multiple cpus and the bug is with the entirety.

============== 2nd July 2007 ===============

1245am: I have run my first pre-emptive multitasking code successfully (after debugging). Just 2 tasks here, the one busy loop echoes "hello " and the other busy loops "world ". Both echoing to the same space as there isnt any windowing yet. These are kernel tasks so this is pre-emptive kernel multitasking.

Further work to be done eg I havent yet implemented task exit, currently tasks must NEVER exit. Various other things also not yet implemented. But the semaphore subproject is complete, the multitasking subproject is almost complete. Enough implemented for the multitasking to function.

This is a fully 64 bit multiprocessor OS. The multitasking and semaphoring is fully multiprocessor and 64 bit but this initial multitasking only utilises one CPU. Some further work before the multitasking utilises all the cpus. This machine has 2 CPUs namely an AMD Turion X2 but the OS will run on up to 256 CPUs which is the limit allowed by the x86 architecture.

Once the multiprocessor multitasking is fully functional I will upload a video of that as multiprocessor multitasking will look quite different from uniprocessor multitasking.

440pm: I have tried running a dozen tasks, a number launched correctly but at some point the system froze up. So I am debugging that, looks like a bug in the semaphore system.

Once there are many tasks then the complexity is severe and bugs are highly likely.

Looks like if I delay before each new task is launched there is no problem, so it seems to be a bug when tasks are launched immediately after each other. On the tower system as I write this I have all 12 tasks running without problem pre-emptively multitasking.

540pm: BUG RESOLVED! all 12 tasks now are running without problem without delayed launches. The bug was a register hidden in a macro which hadnt gotten initialised because the initialisation had been conditionally jumped. That register has now been unhidden and initialised and everything is fine.

I think I will try this on the laptop and then begin making the multitasking multiprocessor. The entire architecture is multiprocessor but there is a bit of further work before the multiprocessorness is complete.

6pm: its all fine on the laptop also, the laptop is dual cpu: AMD Turion X2.

I will take a break now and later begin completing the 64 bit multiprocessor multitasking, should take a few hours to complete as the really difficult stuff is all done.

940pm: it is proving to be more work than I thought,

============== 17th June 2007 ===============

There was a further bug, eventually I found it was a macro with a repeated argument name and the assembler hadnt detected that error. Once the arg name was changed then everything was alright.

Further in-depth testing of the subsystem appears to suggest that everything is fine now. So that completes another subproject.

Some subsystems cannot be tested out except by actual use as they cannot be used in isolation.

I will continue now with other things,

330am: the work continues now ever forwards towards multitasking. I have got several further things done already.

============== 16th June 2007 ===============

The bug I was looking for yesterday is quite a major bug, I am correcting that at the moment: quite a bit of work as it affects a lot of things.

1215am: the bug is now resolved. Have done some very thorough testing of the subsystem and the code is functioning correctly. I need to do some further testing to be really sure. Probably will do that tomorrow.

============== 15th June 2007 ===============

I have been testing out a subsystem I wrote in a recent month and it has a tricky bug, trying to locate the bug at the moment.

============== 14th June 2007 ===============

Working now on all the multitasking subprojects including the semaphores. There are some very complicated theoretical problems,

============== 13th June 2007 ===============

I have completed some architectural subphases towards multitasking, these are running fine on the pre-multitasking system. They are features which are Unix concepts, my OS is completely different from Unix but some Unix ideas such as these are definitive. Just as Xerox's concept of windowing is definitive. All OS's have windows today but each OS does windowing differently.

There could be problems with these new features once the system is multitasking so there may be bugs which will manifest later.

============== 12th June 2007 ===============

Trying out some of the new code I have an impossible bug, have spent a lot of time on it without success.

315am: the bug is resolved. Very very obscure problem,

============== 10th June 2007 ===============

The work now is getting very complicated and involves all the different subprojects of multitasking including semaphores.

3am: the work is getting increasingly complicated. Working on very exciting parts of the architecture.

============== 8th June 2007 ===============

I have completed working through the code I was looking at. That is all running correctly. I am now ready to begin on a major subphase of the project towards multitasking, this next subphase is central supervisor level architecture.

930pm: I am working now on the next subphase, very gradual progress. There are a lot of technical constraints now eg ensuring there arent conflicts between different subsystems. This is the problem where you have the code you want but cannot use it because it conflicts with the caller system. eg the caller may have alignment constraints which arent adhered to.

============== 4-5th June 2007 ===============

I am working my way through the code I did approx March-April, there are many things I have figured out since then so I am making many improvements

5th June: have found some things I forgot to initialise, those things arent referenced yet which is why they didnt lead to problems. Have been stress testing some of the code also. Mainly this work is about carefully reading the code which may or may not be correct. So it isnt the same as reading code which you know to be correct.

============== 2nd June 2007 ===============

I have completed a major early phase of the work towards multitasking. This code functions on both my machines. Most of the multitasking code doesnt function yet and hasnt even been assembled. This phase includes the recent absolute time functionality but also is a lot of other things and is about 6 weeks of work.

I will take a rest from computing and then begin on the next major phase of work.

1130pm: well I am continuing now with the next major phase of work. This in fact continues a subproject I was working on end of March beginning of April. I tried running the code I was working on then and it malfunctioned. I found the bug, it was an incorrect stack. There may be further bugs,

============== 31st May 2007 ===============

I have got a lot done on absolute time functionality. By absolute time I mean say 31st May 2007 03:55:28 or delaying by 2 seconds.

ie synchronising with the time you see on your watch. There are a lot of complicating factors. The central problem with time is that if you try to measure it, it continues moving forwards while you are thinking what to do. Furthermore while thinking an earlier delay may now have elapsed. So the problem is that time doesnt wait, it just continues on and there is a problem of catch-up!

I have basically got completed some important steps towards full functionality. Full functionality wont be there for some time.

============== 25th May 2007 ===============

I am making progress now with time, continuing with the time work I began many months ago mentioned at the end of the web page. I have completed another phase of bootstrapping time. As mentioned recently some time functionality my work was depending on doesnt seem to function on my laptop so I had to workaround that.

I now build my OS using old fashioned MSDOS scripts!

Although I criticise MS I like some of their things eg their script system is very direct. It seems to decide that a file is a script file if it is ???.bat and you can put ordinary commands in that. It also deals with variable script args.

============== 24th May 2007 ===============

Some time functionality I require doesnt function on the laptop, however other time functionality is functioning so I am creating a workaround via that.

later: running into further time problems, trying out some new time things the tower system now malfunctions. This is using legacy 1980s hardware.

============== 23rd May 2007 ===============

Work continues with integrated time delays. There seems to be problems with time and date with the laptop. I am trying to get that to function. The same code is fine on the tower system.

============== 21st - 22nd May 2007 ===============

At the moment I am working on a lot of technical matters, there is an unavoidable problem within one of the subsystems needed for semaphores and I am working around that problem at the moment. The workaround is complicated. I am working also on delay functionality, delay is actually part of the architecture and complicates the design.

============== 13th May 2007 ===============

Recently not many updates as I was away from the internet. I have got some major multitasking functionality implemented. I have only done a small amount of further work on the semaphore subproject. I got digressed into a different subproject of implementing multitasking, something which the semaphores will depend on. That itself was major work.

============== 16th April 2007 ===============

The problems now are getting really impossible. Whichever way I try to do things I reach an impossibility!

I am making gradual progress now, eg there was some code which was only ok if there was no multitasking. I could change that code (A) to be alright if there was multitasking but that depended on something initialised (1) later. I got around that by moving the initialisation earlier. But that initialisation depended on an ABI which wasnt yet initialised. I dealt with that by temporarily initialising (2) the ABI and then deinitialising it. At that point the system depends on a different ABI. With 68k you probably just need 1 ABI, but x86 is unusual and you need different ABIs. Later on some code assumed the initialisation hadnt been done and that needed changing as it now was dependent on the initialisation (1).

I have made those changes but not yet changed the code at (A). The changes at (A) need to function also before multitasking starts. That was another complicating factor. I found a way to do it which isnt yet implemented.

Such problems are very impenetrable because a different way would have led to totally different problems.

Some of the alternatives I looked at I couldnt see how to run the system with. There was an unresolvable circular dependency where one way out was to remove some features! Unfortunately those features really were necessary, eventually I found a different way.

I have tried the above changes with the existing non multitasking system and it runs correctly. If there was any error at all the system would freeze up or reset depending on its mood.

145am: I have got some of the code towards (A) implemented. I have got this debugged now for the non multitasking situation. There is quite a lot to change. Changing the code is straightforward, but I need to do this in batches in case of bugs: I make a backup before each batch and then try that out in the non multitasking case. There was a bug where the text output was freezing up because of an interaction. Trying out the code so far I can see the idea is good. I havent yet tried the code on the laptop, so I'll try that next.

330am: HAHAHA! I found there is a much better way of doing this, I can do (A) with much less coding. Luckily I made backups so I will revert SOME of the files, other changes are fine. The bug above is in code which will be kept.

17th april: the new approach mentioned above has to be done more carefully but is less coding. Today I was optimising some of the changes. One ongoing problem with the current work is that any change at all to any of it can affect everything else. eg some changes mean that a lot of the other code becomes garbage. It is very difficult to create a system which in totality is alright. Each time I read through the new work a lot of the stuff is now garbage. It is like one of those 3D puzzles where you have different irregular shapes which you have to connect up into a regular shape, if anything at all isnt right the entirety disintegrates. As you use one of the shapes you then find it obstructs another shape. Or a rubiks cube where if you move anything a lot of other unintended stuff also moves. A LOT of effort is required to move everything into place with a rubiks cube.

The other problem is that everything is so arbitrary, there are no rules and so many different ways to do things, alternatively sometimes there doesnt seem to be any way to do something.

The reason this code is so tricky is firstly that this is coding tricky hardware: totally asynchronous, secondly that the thing being implemented is very tricky: pre-emptive multitasking s/w + kernel, thirdly that this is the substratum of all software. When you read this page on a browser, the browser runs above an OS. At the centre of that OS is material comparable to what I am synthesizing here.

============== 15th April 2007 ===============

I wasted a lot of time trying out some code, it was 2 very difficult to see bugs, one was a loop where I forgot to put the jmp statement at the end of the loop, the other was that 2 variables were very similarly named and I used the wrong one in a statement. Both are like the "Paris in the the spring" optical illusion where the first "the" is at the end of a line and the second "the" at the start of the next line and you dont see there are 2 "the"s. I read through the code again and again and could not see the problem.

The multitasking project is reaching complexity meltdown, each thing I look at requires ever more work, subproject proliferation, the workload is increasing much faster than the work! One way to deal with this is to leave things out of the design. Eg some features have a time cost, this leads to project definition questions. eg is the project about speed or about features. My project is ultimately somewhere in the middle. I want speed but I also want features. I have to make some difficult decisions as to what the system will be.

A very important part of design is deciding what a system WONT do. Good design is often about what a system DOESNT do, eg an example of good design is AmigaOS not utilising fork() or Modula not using macros. Just because Unix does something doesnt mean its any use!

============== 14th April 2007 ===============

Today I found a disadvantage of the current semaphore code, I am busy working on this problem currently. It is very tricky. The idea I am trying may even be impossible, I havent determined that, if it appears impossible I can try a different idea which I think is possible but not quite as good.

============== 13th April 2007 ===============

I have made a lot of progess towards semaphores. Got some very tricky things implemented, this is the twilight zone of computer science. (twilight zone==television series here! a place where the normal rules dont apply)

I am taking a break now from the semaphores and am doing further work on the memory architecture. Ironically first attempt to upload this froze up WinUAE!

With such things there are 2 bugs which can happen: freeze ups and tasks vanishing! I have to study the code very carefully for such problems. There are so many different interactions that some wierd problems can happen.

The current memory architecture work is very interesting, this work is deep within the architecture and depends on months of work: implementing some cool features.

============== 11th April 2007 ===============

Very busy, gradual progress. Much underway. Progress good, progress slow, progress difficult.

I give here using pseudo 68k asm an illustration of semaphore problems:

simple semaphores can be done thus:

xyz:  tas.b m
      bnz xyz

exclusive_code
...

      move.b #0, m

...
m db 0

tas tests m before setting the top bit. Here a task keeps setting the top bit of m until that results in a change. bnz means the bit was already set so try again. The top bit signifies the semaphore is BUSY. instructions such as tas were invented entirely for implementing semaphores.

That means only 1 task can execute exclusive_code. However there is a problem: if a higher priority task now pre-empts the task and also tries to lock the semaphore both tasks will freeze up. The reason is that the higher priority task loops forever at xyz: and the lower priority one being of lower priority is kept from continuing to eventually free the semaphore. So the higher priority one loops forever on the semaphore and the lower priority one waits forever for the cpu. Here the semaphore is logically correct it achieves exclusion, but the design is bugged as the system freezes up. Thats an unusual problem because nothing has crashed.

Most of the time though the above is fine. Its a 1 percenter, the bug only happens 1% of the time. AmigaOS dealt with this by not using semaphores but switching off multitasking via Forbid() and Permit():

call_forbid
exclusive_code
call_permit

which was optimal for how they did things, but in general has problems eg it is insecure as a prog can freeze the OS via:

call_forbid
rpt:  jmp rpt
call_permit

One way out is the Microsoft XP idea of "program not responding". Essentially give forbid a timeout, so the OS can only freeze up temporarily. The discussion is reaching a point where truth becomes relative. An idea is good on one OS and bad on another. AmigaOS recognised that semaphores were necessary and supplied them with AmigaOS 2.0. The AmigaOS 1.0 RKM by Sassenrath says to use message ports for semaphores. That probably is efficient but eg has the problem that it requires an auxiliary task. Either way the design is becoming complex as semaphores now depend on message ports and require a protocol. Auxiliary tasks are the way for doing certain things, AFA doing semaphores you have to be careful about the auxiliary task depending on semaphores as that is what it is trying to implement.

Whichever way you go, things get complicated, especially if you want to design a good system. Even if the system is correct you can then get user level bugs.

My architecture is completely different, the semaphores will be completely integrated into the design. They co-depend on many other things. I could design the architecture differently so that semaphores arent integrated, but that has its own problem consequences.

It has taken about a year of coding before I can even think about semaphores. So it is like installing lighting in an aeroplane, you first have to make the plane! The lighting depends on the power supply, but the aeroplane control mechanisms also depend on that. You have to make sure there is enough power for the entire journey.

Once multitasking is functioning then I really have an OS, before that it is more of a precursor. Even when multitasking is implemented it will be months before it is fully functional, eg the initial multitasking cannot access hardware.

When the multitasking is functioning I will eventually upload a video of it. Same legacy graphics as the existing video but multitasking, the graphics will be 1980 but the multitasking will be 2010.

============== 9th April 2007 ===============

9pm I have completed coding the component mentioned on the 6th. The work was neverending but I have completed it now. I havent assembled it yet. The code is very complicated and bug prone, but it will be very heavily used so any bugs should soon manifest. The code has been carefully written to avoid various dependencies which could be used. Those dependencies are avoided to avoid circularity: there is a danger of design bugs where the OS freezes up because something is circularly self dependent.

The semaphore subproject is still at an early stage, so the above component is a first step of many towards multitasking.

============== 6th April 2007 ===============

I am making very gradual progress towards multitasking,

950pm: I have just completed coding one subcomponent of one component of the architecture. I havent tried compiling it yet,

later: at least I thought I had completed it, have found several complicating factors. so I am still working on that.

============== 1st April 2007 ===============

4pm: All the different subprojects of multitasking are now merging into one project. so I am now working on all the subprojects simultaneously, some things involve one topic or feature, others another. None will be complete until all are complete. All the features are being fully integrated.

Full integration means that you wont have to implement things by hacked workarounds, instead the things will already be there designed into the system. And there are no workarounds, everything is harmonious.

Anyway, returning to semaphores those wont be complete until everything else is, there is a lot of further work before anything completes.

============== 31th March 2007 ===============

The semaphore work continues, it is very stressful work. Things are gradually progressing. Its a very interesting problem, there are very complicated bugs that can happen.

tautologically I wont be able to test out the semaphores until after multitasking itself is implemented. semaphores are an integral component of my multitasking architecture: my OS cannot function without them.

============== 30th March 2007 ===============

Finding a lot of theoretical problems, some of the existing code wont cope with these so I am having to redo various things.

As I progress towards multitasking the problems get increasingly difficult.

I am working on semaphores at the moment...

semaphores are very tricky, looks like another phase or subproject...

semaphores are inextricably linked to multitasking, the 2 problems are inseperable. So the semaphores will be designed and implemented in tandem with multitasking in general. Very difficult to be sure there are no bugs with the mechanism.

============== 29th March 2007 ===============

I am working on further memory architecture work, I had an interesting crash because I forgot that x86 stacks start at the far end.

============== 28th March 2007 ===============

I am working on a very difficult component of the multitasking architecture. This is supervisor level work. This component is the beginning of a new phase towards multitasking. The component has to deal with many different problems. Also the kernel is pre-emptive which complicates things further. AmigaOS was the first pre-emptive kernel.

============== 25th March 2007 ===============

Last thing yesterday I realised there was a better way of dealing with several x86 specific memory constraints. Several different workaround mechanisms could be replaced by 1 general scheme which also allows me to deal with new constraints or remove existing constraints by changing build parameters.

I have completed implementing that and it seems to be functioning fine. The new code is much easier to maintain.

When I make extensive changes I always backup the preceding system first in case the new work runs into problems.

I am ready now to begin the next phase of work towards multitasking, this is further memory architecture work.

1am: I am currently doing some low level reconfiguring of the hardware required by the plan. Without the reconfiguring the system will not function.

later: I've been trying out the code so far, x86 is doing something different from what I thought, so I have to figure a way around that.

============== 24th March 2007 ===============

I have been trying out the code so far. This has a bug which I am working on at the moment.

1025pm: that bug is resolved, I forgot to set some field. Fine on the tower, I am about to try it on the laptop. (later: that went fine).

Before continuing I need to make some improvements to the existing code.

530am: I have made a lot of improvements to the existing memory architecture (and several bugs getting there). Some of the improvements are x86 specific and some are general. Some of the x86 specific optimisations can probably be ported to specific requirements of other cpus. (would need to be redone in C first). The code avoids hard wired assumptions. There is a lot of further work to be done on the memory architecture.

============== 20th March 2007 ===============

I am right at the centre of the OS now, centre of the multitasking. A lot of further work before the multitasking will function. This is the most difficult part of the whole project. The code is still unitasking eg the recent video is unitasking.

No graphics at all yet, just the legacy ascii graphics. If you are very observant you will see that the snapshot of the laptop has slightly different fonts from the video of the tower. The 2 mobos have implemented the same fonts slightly differently. eg the 0's are different. I have the graphics hardware under control but that code isnt integrated into the OS. So if you were hoping to see some windowing in the video, there isnt any. It hasnt even been coded yet! The graphics system will use the CPU as a GPU.

The OS so far is completely written in x86 asm. At some point I will start using C, and some of the existing code will be redone in C. But a lot of things have to be done in asm.

The initial multitasking will have many features but wont be fully featured till later. The multitasking will gradually become fully featured. The full feature set is too complex to implement in one go, it will be arrived at in phases. The later features are designed in right at the start but implemented later. To reach multitasking is itself a major bootstrap consisting of many subprojects. eg the lowest level of the memory system took approx a month to develop. The higher levels of the memory system are still being developed ongoingly.

As an example the initial multitasking wont have access to any hardware except to echo legacy text to the screen as in that video. Task level access to hardware is major work.

I see Vista is now in the shops, I'm not at all worried about Vista though I will eventually try out Vista to evaluate it. Microsoft wont wait for any competitor! Their presentation is full of smiling dancing young actors pretending to be Vista users. with oiled teeth and ants in their shoes. They certainly know how to market thats for sure.

But if you switch on your television and watch the adverts you will see that all companies hugely hype their wares with actors pretending to be customers.

============== 18th March 2007 ===============

At the top of the webpage is a photo after the OS has echoed the first sector of the HD via a 28-bit sector number DMA ATA read.

I'll try and upload a video of a subset of the code in action. I will only show a small enough subset that doesnt reveal too much of the gameplan. Currently the graphics is entirely legacy hardware 16 colour ascii 80x25 text. (IIRC it is 16 colour, something low anyhow).

You will notice the numbers are different from those mentioned on 5th March, that is because those are for the tower and these are for the laptop.

I have now made a video of my IDE DMA code in action, DOWNLOAD THE VIDEO

The video shows some HD query code then reading the first sector. After that I press some keys which appear and move the mouse + press and release buttons. The mouse coordinates are echoed and the button changes are echoed. I press the left button down, right down, left up, right up. This is for my tower system and you will see the numbers are those mentioned on 5th March.

Although the graphics is still very primitive I guarantee that the OS underway is very advanced.

Most of the work isnt visible, the visible stuff is the tip of the iceberg. Right now I am working on some really tricky code towards multitasking. This sub project gets increasingly difficult as it proceeds.

============== 15th March 2007 ===============

I am coding one of the phases towards multitasking, this code has to deal with various future features such as security, task removal, fork() and others. Security is preventing things like viruses, spyware etc.

The security architecture is entirely my own design and is not based in any way on any other system. Security is a very far reaching topic. The security design is ongoing work and consists of different phases each of which is a subproject. The initial phase I am coding now and was arrived at over recent months.

The reason Windows has hundreds of viruses and other malware is because it is a mainframe OS which was never intended for PCs. Bill Gates paid the chief programmer of VMS to reimplement it on PC's. Windows has painted itself into a corner surrounded by viruses. viruses are at the centre of the design of Windows.

My OS is not a reimplementation, the entire design is brand new and as such I avoid all the problems of all the mainstream OS's.

============== 13th March 2007 ===============

Major work is now underway working towards multitasking. This is a brand new architecture with major features eg security, crash protection, clean task removal, pre-emptive kernel, fork() (as far as is feasible), and many others. The OS design will be as good as any on the market, the 3 comparison OS's are 68k-AmigaOS, Unix and Windows. Of the competitors AmigaOS has the cleanest and most efficient design, Unix probably has the most features of these 3 and is the least usable. Windows has the most attention and is certainly the most aggressively developed. However an Amiga 1200 outdoes say XP in many ways, AmigaOS is a much more general design eg AmigaOS is the most asynchronous OS ever created even today. AmigaOS has the most advanced windowing architecture, Windows windowing has useful features but the actual architecture has many limits.

Because there are so many deep level features the work is quite severe. Some of the features will be worked on later. Remember I am attempting to outdo Unix which has stood the test of time since the 1970s. AmigaOS also has stood the test of time of 20 years. AFAIK Windows is a reimplementation of a mainframe OS called VMS, the disadvantage of that is a PC is a totally different hardware paradigm from a mainframe. Its also why Windows is so severely complicated. Unix is also really a mainframe OS thats why it has the user-group-other file protection flags. A Unix system will have say 200 users who login through terminals, 5 of those users might be a team working on some project and their shared files will be group readable. A file is group writable if anyone in the team can modify the file, the filesystem creates an exclusive write lock so that 2 people cannot write the file at the same time.

Windows 95 made the astonishing assertion that it was a man century of work my project will prove that is false.

programmers will be discouraged from using fork(), instead there will be other ways for user programs to multitask. Example problem with fork() is if a program is writing a CD-R and right in the middle of that does fork(), 2 programs are now writing the same CD-R which is clearly a problem.

you wont be able to interfere with the OS like you can on AmigaOS, AmigaOS is unique in that respect. advantage: users can customise the OS, disadvantage: so can viruses.

============== 9th March 2007 ===============

I've completed writing and debugging a very initial batch of code towards major OS functionality,

============== 6th March 2007 ===============

Doing a lot of coding now, major stuff.

It may not be clear what the recent PATA info was about: what I achieved was making the PATA code BIOS independent, ie it now doesnt use the BIOS: thats an improvement to the system. See (A) in the project summary and (B) also.

============== 5th March 2007 ===============

further to yesterday, I was going to implement 28 bit sector numbers for PATA but as I started on this I realised I must have already implemented it. Reading the code more carefully I found it was already done: I had implemented 28 bit DMA but NOT 28 bit polled code. The code I was trying yesterday was in fact polled 48 bit code. Literally 4 lines of code later and the tower 28 bit is reading sector 0 correctly. Its on the other screen right now, the last 18 bytes being:

00 00 c1 ff 07 ef ff ff 40 94 45 04 c0 11 0b 05 55 aa

If you know enough about PC's you can figure out from that some info about the partitions. eg the 5th byte along is 07 which would appear to mean that the 4th top level partition is NTFS. The 4 bytes before 55aa tell you the partition size, 050b11c0 which is about 5 x 2^24 x 2^9 bytes ie 5 x 2^33 = 40 x 2^30 = 40G, and XP says 40.35GB. which is about half of the drive. This type of interpretation is important to determine if hardware code is correct.

I know the read is correct if it ends in 55 aa as that is the signature of sector 0 on PC's.

I am continuing now with the main OS work towards major functionality. Implementing the OS features people wish for in forums!

If you dont like what I am doing please email me and say why,

============== 4th March 2007 ===============

I have been revisiting the PATA HD code. I want to make the HD code fully BIOS independent. It was only some time after I had written the code that I figured out how to remove the BIOS dependencies. I didnt do that then as there were various complicating factors.

Anyway I have enough things now under control that I can make the HD code BIOS independent. I have been working on that and have now completed that code, but there are 2 problems: the tower system is reading the first sector but the bytes dont look correct. The laptop HD initialisation doesnt directly function, I know why and am working on that right now. Once it functions I will see whether it reads the correct bytes.

As regards fork() it seems the function is not intended for the more general situations I was considering.

later: both HD problems are now explained: the tower system currently uses an 80G HD which doesnt support 48-bit sector numbers. The current code just uses 48-bit sector numbers, so the read there was a no-op. The laptop HD needed to be reconfigured, it then did read sector 0 correctly. I know if the read is correct because sector 0 on a PC HD should end in the bytes 55, aa.

I think I have already written somewhere 28-bit sector number code, looks like I'll need to get that functioning in order to be able to test out code on the tower. This isnt urgent, main thing is the 48 bit code is correct. 28-bit sectors gets you up to 2^28 * 2^9 == 2^37 ==128G bytes. Drives bigger than 128G should therefore support 48-bit.

when the laptop floppy takes ages reading, I try to music conduct the floppy drive sounds. The sound of machines is a form of music, eg musical instruments are themselves machines. People and animals are also machines, eg horse drawn carriages and police dogs.

even later: looks like I probably havent implemented 28 bit sector number code. I have been working on so many subprojects that I have lost track of exactly what I have or havent implemented. I sometimes find things I forgot I had implemented. I think at the time I wasnt so interested in laptops as 28 bit is more for laptops with 80G drives. Its on my todo list then to implement 28 bit. ironically this laptop has a 48 bit drive (120G) and the tower has a 28 bit one. That's because I decided to go for a smaller system HD for the tower so I could move my stuff between computers by recabling. OTOH I went for a big laptop drive as it is inconvenient to use external drives on a laptop. Now that I've learnt how cool laptops are I will ensure my system runs on modern laptops with 28 bit or higher drives.

The HD code more generally will require a lot of further work,

============== 2nd March 2007 ===============

I have been studying fork() to see if I should make my OS fork() compatible.

I never use fork() myself. AmigaOS's CreateTask() is a much better function. fork() is only necessary if you want to port Unix progs to your system. fork() isnt recommended.

Having fork() compatibility is a severe design constraint as you literally have to design the whole system to allow fork() to be possible. For the moment I wont decide and will just see what the consequences are.

fork() is certainly an interesting idea, it is the thing of the universe splitting into 2 paths eg the one path you bought an Amiga and the other you bought an Atari ST. However being interesting doesnt make it good programming, making an optical mouse the size of a car is interesting but not practical engineering unless Gulliver is on his way. eg if the universe can split into 2 paths that hugely increases the dimensionality of the universe which makes it more difficult to implement. interdimensional memory isnt cheap...

Note that I dont yet have multitasking, fork() is being looked at as that constrains the entire OS design. To see the severity of fork() think of UAE as a program above Unix, what happens if you put a fork() statement?

UAE should then split into 2 UAEs, but how exactly? eg what happens to the interactions with the hardware?

Ultimately the objective is to create a good system, and if fork() gets in the way of that objective it will be removed as a constraint.

Trying out some code there was a bug with fantastic consequences, I put a line of asm in the wrong place and there were fantastic colour effects, much more interesting than the usual correct code!

The colour effects are on the other screen right now,

============== 28th Feb 2007 ===============

I have got several low level hardware things done today. As I write this I have some code running on the other screen.

things are progressing very well,

The code above eventually reset the machine which was a bug, various further changes and the code seems to be alright. I have the newer code now running on the other screen. I'll finish the session by trying the same code on the laptop.

Some things are slightly different on the laptop, this is where betatesting is necessary. However it is much too early for betatesting.

============== 26th-27th Feb 2007 ===============

I've got some very initial code now functioning. The bugs mentioned below have been resolved, took a lot of work to do. I have made some improvements also to the supervisor level of the OS. A long standing inelegance has been smoothed.

This is a very initial step towards the app level of the OS.

I've also improved my work desk! I now have the tower monitor to the right of the laptop so I can I have both machines in view simultaneously: with the "OS" on the right monitor and the source code asm on the left monitor!

One unusual thing is these 2 machines use different hardware fonts, the monitor on the right has 0's with a dot in the middle whereas the left monitor has 0's with a diagonal line.

27th: I have improved the mouse code (that was on my TODO list for many months). The mouse code is not yet app level, I will work on that later. I will continue now with the app level of the OS.

The work now is very difficult, so at the moment I'm not doing any coding but am studying the problems. At some point soon I probably need to resume work on the compiler so that the further development of the OS can be done in C.

============== 22nd-26th Feb 2007 ===============

Working on very difficult OS architectural problems...

23.feb: found a very complicated harmless bug.

If you want more specific info on what I am doing and why, email me with a specific or general question,

I can give more info in an email than on the website.

24.feb: I'm doing gruntwork programming for some interesting new functionality. This is very initial work on the app level of the OS. And yes it will be a pre-emptive multitasking OS,

25.feb: there's a bug now in the debug code!

26.feb: several bugs actually. I've got some of them resolved now, involved some supervisor level coding.

============== 21st Feb 2007 ===============

I am making gradual progress on the hardware subproject. I'll get a bit more done and then resume work on the OS: the next steps with the OS are quite major things,

When I say "hardware" I actually mean software!

Note that a lot of actual hardware is itself software eg x86 is in fact software above a RISC core. Years ago someone made an interesting remark that "hardware is software".

============== 18th Feb 2007 ===============

I've just tried for the first time the OS ported to the laptop back on the tower system and it functioned fine first time. This will greatly speed up progress as I can now develop on the laptop but test the work on the tower.

I am working on the first steps of a hardware subproject.

============== 16th Feb 2007 ===============

First update for a month, in that time I have bought a cool laptop and have ported my OS project fully to the laptop. All bugs mentioned in previous updates have now been resolved: there were many bugs and some really tricky x86 asm problems.

Not only have I ported the OS to the laptop but I am working on a lot of new things on the laptop.

============== 15-16th Jan 2007 ===============

right at the end of the day I've found a function in the boot code which contains several bugs some of which directly relate to the supervisor level bug. The code looks correct but it isnt. It took some work to determine that it wasnt correct.

I'll try to correct the function tomorrow.

I am buying a laptop now so I can continue my projects when I dont have access to my tower system. I will probably try and get the code running on the laptop though that is trickier as laptops dont leave much room to manoevre.

16th jan 7pm: I am working on the problem. This code is 16 bit asm which is very fiddly. The bug happened because I was trying out new assembler features and no problem manifested at the time.

I have found now the cause of a lot of the problems.

11pm: I've corrected the function but the problem still manifests. I have to continue this tomorrow.

============== 14th Jan 2007 ===============

Now I've completed the port to 68k-AmigaOS of AFPL Ghostscript 8.54 I am restarting work on the supervisor bug mentioned below. It is an x86 specific problem. I only have a few days of time window so I will focus mainly on this bug.

============== 29th November 2006 ===============

2am: I didnt get too much done today (Tuesday) as I had various non computing things to do. I have made a small amount of further progress on the supervisor bug. I now understand generally what the problem is but it will take some work to resolve it. I wont get much done tomorrow (Wednesday) either.

============== 28th November 2006 ===============

I realised some enhancement that would improve the early compiler phases, so I decided to implement that before I forgot. The enhancement is implemented now and seems to be functioning fine.

I am continuing with the supervisor level bug. This is a very complicated bug and I need to resolve it in order to continue the OS bootstrap.

8am: I have made a certain amount of progress with the bug. Not fully there yet as there is some further inexplicable crashing.

============== 26th November 2006 ===============

The supervisor level bug I'm looking at is very tricky. I am making some progress on the problem, however it isnt resolved yet.

============== 25th November 2006 ===============

As the C pre processor subproject is complete I have moved that info to the project summary at the end of the webpage. Thats another chapter of the project complete.

I am now working on the supervisor level OS bug I was looking at on 17th October.

I will take a holiday from computing in December. I will begin on the C compiler in 2007.

============== 21st September 2006 ===============

The low level OS component is now fully debugged. All tests are running correctly. So I regard this subproject as complete. Perhaps there are undetected bugs these will be worked on as and when I find them.

I have thus completed 2 immensely powerful subsystems: the low level OS component and phase 2 of the compiler project.

I will now make various improvements to the existing OS.

============== 25th November 2006 project summary ===============

This is an updated version of the 21st September 2006 update which was an update of the 23rd July 2006 summary.

Here is a summary of major developments: the major hardware success was getting DMA PCI control of hard disks functioning. That took months of work to figure out, I can now read and write via DMA, ATA drives via 28 bit and 48 bit sector numbers. 28 bit is used for drives which havent implemented 48 bit. Drives below 128G dont need to implement 48 bit, some dont. I have now completed the second phase of my compiler project, this is the second of approx 8 phases towards a 64 bit C compiler, this second phase took 1.5 months to complete. It is one of the most difficult phases of the compiler project. I plan to create a big endian x86 option as well.

I have a very low level PS/2 mouse driver functioning, I've implemented mouse position but no mouse pointer yet. So far I've been unable to reconfigure the mouse though I have managed to initialise the mouse. I can do the reconfiguring in software as the initialisation defaults give 16 pixels per mm resolution which is 406.4dpi so its not a major problem. I have PS/2 keyboard support, though no driver yet. Any keyboard functionality I need I can implement by hacking directly eg I've implemented WaitForChar(), this polls till the key is pressed. It is better to wait by not polling, however it will be some time before I can implement that.

I have various low level time features implemented, eg code to read and set the time and date and can do timeout polls of hardware eg timeout after 3 seconds. Without time hardware you can only poll via a counter and the code may fail on a faster CPU. I have a low level DMA floppy disk driver running: I havent yet implemented proper error handling. Dealing with error handling is a gruntwork subproject. Floppy disks have various complicating factors eg if you keep reading from a disk the head gradually becomes misaligned. This means that some errors are not real errors and you need to reset the head and try again. That is something I will need to code later.

I have a lot of low level system hardware functioning and have gradually developed my own efficient low level reconfiguration of the system hardware eg mobo hardware.

(A)By understanding the low level hardware I have managed to find a way around various BIOS dependencies, eg I can now fully bootstrap ATA PCI DMA without the BIOS. Not yet fully implemented that as it requires some work but know how to. (2007.3.6: now done see (B) below)

The current bootstrap uses legacy x86 text graphics. x86 has text rendering in legacy hardware, this gives you the well known standard x86 80 x 25 text. However a modern OS cannot use that, its very useful though for the bootstrap as it allows you to work on the CPU first. I have truecolour low level graphics functioning but completely unintegrated. So far only done graphics in 16 bit, that only gives you access to a few cm at the top of the screen as the rest of the screen is beyond the memory range of 16 bit. That must be why they used text graphics hardware, its the only way to do text in 16 bit. Integrating modern graphics is one of the next major subprojects eg I will have to implement my own fonts. I will use the CPU as a GPU as that gives me full compatibility with all graphics cards.

The filesystem project has been stalling, for the moment I am working on other things. I have tidied up the existing code and the compiler project has borrowed interesting subsystems from the filesystem project especially the memory debug system. I have also generated various plans for what to do next with the filesystem. The FS project influences all the later subprojects as I invented many mechanisms for the FS which are generally applicable.

There are endless miscellaneous subprojects I've worked on, eg I wrote my own generic ELF loader in c and have designed various things to implement later. Some subprojects have to be postponed as they depend on things not implemented yet. eg the filesystem depends on a 64 bit C compiler and an ATA driver. I have now enough ATA functionality implemented to supply the required ATA driver. However I regard other problems as more urgent.

21st September: I have just now completed a major low level OS kernel component which has taken 2 months to develop. This is code that normally I would only do in C, but here it has been done entirely in asm: mainly because the C compiler wont be usable for quite some time. Probably it would have been better done in C, its just that I was impatient.

Early phases of the compiler project are complete which are not C specific.

22nd Nov 2006: C pre processor is complete. This has not been thoroughly tested. It is a fully functioning C pre processor. It doesnt yet have support for floating point numbers, otherwise it is complete. Floating point support will be straightforward but I need to study what is involved. The C pre processor has various enhancements over the standard. More on these later.

Although the work so far looks very chaotic that is just temporary, same way a building site is total chaos. The chaos is because of the high level of complexity. When I say temporary that could be many months.

Feb 2007: the OS now has been ported to my new laptop and runs on both machines. The work now is focussing on the synthesis of OS architecture gradually manufacturing a pre-emptive multitasking system.

(B)5th March 2007: The ATA PCI DMA code is now fully independent of the BIOS, see (A) above

2nd July 2007: first successful kernel level pre-emptive multitasking has been run. This is an initial version of pre-emptive multitasking, the subproject isnt complete eg there isnt any task exit implemented yet so tasks currently must never exit. The semaphore subproject is complete and fully functional.

The OS is completely 64 bit and is fully multiprocessor x86. The multiprocessor code is tested on a dual CPU AMD Turion X2 and will run on up to 256 multicpus as that is the limit of the x86 architecture.

2nd August 2007 5am: the new pre-emptive multitasking architecture is now functioning correctly. This tries wherever possible to relaunch tasks to the same cpu in order to optimise cache usage. Sometimes it is more optimal to relaunch to a different cpu: a very complicated heuristic optimisation algorithm is used. This is much more complex than the original architecture. This makes the OS super modern.

The plan now is to implement a C++ compiler instead of a C compiler.

9th April 2008: major changes of plans in the months since the above was written, in October 2007 I abandoned the C++ project as C++ is not what I want and too time consuming to implement. No point spending many months implementing functionality I never plan to use! Instead I have since then been designing my own language from scratch which does what I want and implementing the compiler for that. I have just today (9th April 2008) completed implementing the code that checks for type equality and the compiler for this new language is probably 75% complete. It is a very difficult project.

Once the compiler is complete the remainder of the OS project will be written using my own language. C was invented to implement Unix with around 1978 and here in 2008 I am inventing my own language to implement my own OS with!

My language is a completely different paradigm from C++, and will give me the functionality that the OS coding requires. The type system has comparable power to C++ templates but is geared completely differently.