 So, I think it's time for another live coding session. And just for a change, I'm going to be writing an assembler. I do do things that aren't assemblers on it. It's just that assemblers are simple self-contained tasks that work well on camera. So the context is that not long ago, somebody submitted, very kindly, most of a PDP11 back-end for CalGoal, my programming language. This is still a work in progress, but I have been integrating it into the CalGoal tool chain. And one of the things that you have to have, along with a compiler back-end, is a assembler and linker to turn the resulting assembler source into something that you can actually run. Now, I've been looking at the standard PDP11 tools for this, which is macro11 for the assembler, and I think it's a tool called P-Link for the linker. But they are kind of overkill for what we want for CalGoal. And I think that it would be cool to write my own simplified assembler for the PDP11. CalGoal doesn't need a full-scale linker. It generates a single assembler output file and expects this to turn into a binary that you can actually execute. So by writing our own assembler, we can do without dependencies on two different external tools. This is also interesting because the PDP11 is somewhat more complex than the 8080, which is what I currently have an assembler for. So by writing an assembler for the PDP11, I then get to figure out how to do CISC architectures in my assembler framework. So some background. The PDP11 is a very old architecture dating back from the 1970s. It's a 16-bit system which uses byte-addressed memory. So it's not actually that different from many other 16-bit systems like the 8088 or the MSP430. However, it's got a very good reputation for being orthogonally designed. The instructions are very regular, even though it's a CISC system. And it's still got a following today because it's apparently extremely nice to write code for. Now I've never actually done this, so it will be very interesting to learn about. It is one of the most influential computer architectures of all time. So supporting it would be a really nice thing to have. So the way it works is every instruction, and let me find my notes document, every instruction is a 16-bit word which may be followed by up to two more 16-bit words which contain the parameters. Now this is in hexadecimal, but traditionally the PDP11 does everything in octal, which I'm not particularly good at, so I believe that's five, six digits per word. So valid instructions will either look like this, like this, or like this. How many extension words there are depends on the addressing modes of the instructions encoded into the first word. Now this is what makes it rather different from the architecture I currently have an assembler for, which is the 8080. On the 8080, every instruction maps onto a single opcode. I can't remember the numbers, but let's just say it's these. And every instruction you know from the opcode what parameters it's going to take. So you know that MVI is always going to take a register followed by a parameter. LXI is always going to take a register followed by a 16-bit parameter. ADC is always followed by a register, let me see for example. So you know up front at the time you read the instruction from the assembler source file, what parameters you're going to be expecting and also what bytes you need to emit. This makes assembling it pretty straightforward because you can just work from left to right. Now the PDP11, because it uses multiple addressing modes and extension words, is a bit more complicated. For example, if you have this instruction, this will be a single output word. This instruction is going to be an instruction word plus an extension word. This instruction is going to be an instruction word followed by two extension words. How many extension words to expect is encoded into the bottom for digits of the instruction word so that you can't, when you're assembling it, you don't know what instruction words to emit until after you've read the two parameters. This makes assembling it rather more complicated. So I have taken a copy of the 8088 assembler, which is here, and set up the builds. So as with last time, it's doing auto builds over here. So if you save the file, you can see Cowgirl recompile it. I'm now generating x86 assembly rather than sorry x86 binaries rather than 88 binaries just for simplicity. I can then run it using that was not supposed to happen. Let me just make a, let's just see if it's just a simple write. That is definitely not supposed to happen. Let me figure out what's going on there and get back to you. Okay, that was a little bit clumsy in my part. I was in fact using the 64-bit tool chain to assemble and link the binary rather than the 32-bit tool chain resulting in garbage binaries. Anyway, I have the build system set up. So if I save the file, it rebuilds and I get something I can run. I have a source file which I can write PDP11 machine code assembly into. I can then try running it with the assembler. This of course won't work because it's expecting 8080 assembler and for verification, I also have a PDP11 disassembler so that I can then disassemble the result and see whether it looks valid or not. I also have here a disassembly of the original PDP11 assembler. Unfortunately, I don't have the source code for it but this gives me a wide array of test instructions that I can try. Unfortunately, I don't have everything set up yet for a end-to-end simulation test like I did with the 8080 assembler. That should come later so this is all going to have to bench tested but I should have enough here to actually try it. So here in the disassembly you can see this is the address in octal instruction word, first extension word. You can see that this jump instruction takes a single parameter which is encoded into the extension word here. You notice that these values are different. I'll get on to that later. It's using relative addressing modes. Let me see if I can find a absolute instruction. Here we go. Here's a two opcode sub instruction. This is subtracting this constant value which you can see encoded into the first extension word from the value at this address which is encoded into the second extension word. That is opcode 16. If I go over here to the handy table in Wikipedia, where's the set of opcodes? One six is indeed sub. We've got two seven for the I believe that's the source yes which is two seven which is immediate although that there is in fact this is in fact done using some cleverness I will get to later and the second one is six seven which is relative. So this is this first value is an immediate operand and this second value is a relative addressed operand. So how are we going to start? Well let's go to the assembler and start ripping out the 8080 stuff. We have here we have the big table of opcodes which I'm going to have to copy from Wikipedia. So let's just remove all of these except for the meta operations. I'll just remind myself oh yes and we also have we need values for the registers as well. The pdp11 is eight registers numbered from zero to seven. Register seven is the program counter and register six is the stack pointer. So in fact let's just stick those in now just like on the 8080 we're just going to implement these as labels which resolve to numbers very very simple two three four five six seven seven six five four three two one zero and we want aliases for stack pointer and program counter. Okay now the pdp11 actually has where we got instructions a few different types of instruction we've got double operand instructions like this sub here single operand instructions like jsr you've got one right here which takes a single that's not working quite how I expect let's see oh oh zero zero yeah zero zero four five six seven so ah right this is using a four byte opcode and the bottom octal digit specifies the link register the so that's zero zero four five that's using five as the link register I will talk about that later uh and six seven is the addressing mode of the single parameter which is relative it's the same addressing mode that we were using for here it's a relative address uh in fact j s r e m t etc are specials that was not a very good example clear is a better one there should be a clear there's a clear it's four digits of opcode zero zero five zero followed by two digits of addressing mode and that can take one uh extension word so uh we've got these we've got the special forms for j s r and e m t we've got conditional instructions which are encoded quite differently they take a eight bit offset in the bottom of the instruction this allows you to do small branches uh without needing an extension word and you have the others including condition code registers etc I will have to look into these in more detail I actually thought there was a full list in Wikipedia but it looks like there isn't so I'm going to have to look those up now the way we're going to implement these is our symbol table here has got a field for the callback which tells the assembler what it is you actually want to do with the instruction and a field for the opcode this is a 16 bit field we're only using eight bits for the 8080 so let's go with a just for demonstration let's try a double operand instruction mauve is really useful so mauve the opcode is octal no one oh oh oh the bottom two the bottom four digits will be filled in by the callback and this is going to be a double operand instruction so this callback is actually going to be used by all instructions that follow this form it will read in the two parameters patch the addressing modes into the mauve into the opcode and omit it so here is mauve b so we should be able to use the same callback for all double operand instructions and likewise we are going to do the same thing for single operand instructions here we've got clear clear is zero zero five zero followed by two digits like this so we're now going to have to implement these now that of course this won't assemble compile rather because we haven't implemented these things but let's take out the 8080 stuff first now we do need to maintain the symbols for the special instructions so we've got the operators used by the instruction evaluation we've got the special forms used for emitting a constant eq else and if this is a register this is a register these are all simple instructions that's a register and xi is special mod is an operator not is an operator gorg is special psw is a register these are operators so it's a register title is special like so and here we have the actual implementations they should be in a sensible order at eq set end if else if db ds dw rp what is rp i can't remember what that is that wants oh it seems to have cut it out therefore it's not going to be useful org okay and here we have the callbacks for the actual instructions and we're going to want to remove these but i might want to just comment them out for now so i can refer to them later so you know once since i've worked on the assembler so trying to remember what the comment instruction is and let's move this up next to if and friends belongs right up one up two so this is going to expect a single operand and then it's going to do something this however it's going to expect two operands separated with a comma yeah the way i worked this with the 8080 is that when you read the expression it returns the terminator so i'm going to have to make expect operand do the same thing so expect operand will read an operand and then if you don't get the end of instruction it produces an error read operand will read an operand and return the terminating token so in this case it has to be a comma so this will read two operands and do something and of course we don't have expect operand now up here should be yep here's the expression reader and here is indeed expect expression and bad separator so we're going to have a read operand which is going to return a token do i have a type for a token i do and we have expect operand which returns a token straightforward expected a single operand 8060 requires output operands right this does not actually return the token right and that assembles the tricky part is going to be reading the operand the operand actually is going to need to return both the addressing mode and whatever value needs to go into the extension word so where are our addressing modes the pdp11 has eight addressing modes plus some extra addressing modes that are really syntactic sugar around these addressing modes your eight addressing modes are work on a register work on the value pointed to by a register work on the value pointed to by the register with a post increment work on the value pointed to by the address pointed to by a register so it in fact dereferences the register and then it dereferences the memory location in the register this is useful if you are working on pointers stored in memory and this has a post increment form we have the same for pre decrement we have an indexed form where the where you take the value in the register you add on a constant and the result is the address of the operand and you have the double dereferencing version where the result of the addition expression is an address in memory now the extra addressing modes work by using the program counter or the stack pointer with one of these and I will demonstrate how that works so we have we load a value into a we load an immediate value into a register that is actually encoded as more or less but the instruction that you get in memory is this the with this form the when instruction is executed the instruction word itself is read and then the program counter advances to the instruction afterwards then the parameter is evaluated this is the post decrement form referring to pc so that reads the value at pc which is 1234 and then advances program counter so that even though there is no actual explicit form that does this you get the same effect by using one of the existing addressing modes and the other addressing modes work in a similar way we've got absolute addressing and relative addressing these are variants of these two addressing modes here from the point of view of our assembler we're going to have to know how to read these and then generate the appropriate addressing mode and extension word this is where the work is going to happen likewise you also have similar syntactic sugar around pushers and pops so what we're going to have to do is to figure out which one of these we're looking at this is going to be moderately annoying to be honest we have an expression parser this can read any arithmetic expression that the assembler knows how to understand the thing is since registers our expressions this is an expression this is an expression these both return zero this is also an expression that returns zero so we're going to have to be a little bit clever about how we do this for example just reading an expression means that these two values will produce the same result we can't tell the difference because this second value is just a parenthesized expression it's equivalent to you know this you are allowed to do things like this by the way that gives r1 so we are going to have to look at the first character of the expression to see is it a open parenthesis is it an at sign is it a minus sign is it a hash sign and then disambiguate based on there now notice that we have this form this is supposed to have values like do values like this but this first item is a number so it is actually legal to do this this allows you to have parenthesized expressions in the first form so in terms of the things we have to be able to pass if the if the first character is a open parenthesis then it must be one of these one of these because the parameter at the beginning is parenthesized one of these no sorry not one of these because we know that the first character is a parenthesis which won't apply for this in fact i think it is just those two that's oh yes and one of these so the algorithm we're going to have to use here is look at the first character if it is a parenthesis then read it then look at the next character if it's another parenthesis read that we can't actually tell the difference between one of these and one of these doing that that's a bit irritating we could consume the leading parenthesis yeah we can consume the leading parenthesis read the expression that will go up to here i think that doesn't help so this addressing mode this addressing mode only exists as indexed based on pc this is used a lot so it's important we get that right because our registers are just symbols we can't identify we don't know the difference between a simple arithmetic value or a register i wonder if we're going to have to change that somehow it it's entirely possible to just budget that if the value that we read is seven or below then it must be a register but you are the the pdp11 uses a segmented architecture if you look at our disassembly here you can see that the addresses start at zero that is actually where they're loaded in memory so if you try to jump to any of these then you the assembler can't tell the difference between a natural address and an arithmetic value uh macro 11 the stock assembler uses percent signs rather than rs to identify registers which allow would allow us to distinguish between them at assembly time the other thing we can do is to change the characters used for indirection to something other than parentheses square bracket is a common choice that way we can distinguish between grouping parentheses and actual indirection but i'd rather like this to be as uh standard as possible just looking to see if i've got any macro 11 source code handy you don't think i do is there anything there no that's just the there's no actual documentation for macro 11 here so i think i'm going actually going to have to i think i'm going to have to take the registers out of the symbol table let me just take a quick look at the code generator to see what it produces for actual register values yeah it produces rs so that is what the author of the code generator expected we can read well we at pass time we can distinguish between uh registers and other things based on the callback this requires us to peek the next token and check the callback this means that you now cannot work on registers and expressions but i think that's fair yeah i think we're going to have to do that where are our callbacks we already have a yeah end callback here is is one of these it doesn't actually do anything when you run it so let's put here like so in fact when this will be executed if it sees one of these as the instruction parameter so we can actually put a this should never happen so let's put a and this is going to want to be up here above read operand actually okay that works so when we read the operand we need to peek the next token if it is a register we know it's this form this that's this form if it's not a register then we know it must be this form or something else if you try to use a register in an expression you'll get an error and we'll have to check that okay let's look into writing some code so we want to peek the next token did i actually implement peek i hope so i may have done this as a unget yeah we've got push no that's a pushed character i did not implement peek right let's implement peek we're actually going to do this as a so push c writing too much c if push c is not zero then push token is not zero and then token is pushed token okay now when you read a token it will actually set a number of global variables to to contain the things that the token points at so you can only push a token immediately after reading another token but that should be fine yeah it does stuff like resolve the symbol so where is our read operand okay if at this point the token can be it can be an open parenthesis which means that we're looking at an expression or a parenthesized register so at this point we want to look at the next token if the token is a do i have a value for token identifier if it is an identifier and we're testing here to see if it's a register token symbol will be pointing at the symbol yep if the token inside the parentheses is an identifier and it's a register then this must be a parenthesized register otherwise we need to read the expression so we push the token back we read the expression which must be an expression and then we need to read in yes and this must be followed by a closed parenthesis if read expression is not equal to a closed parenthesis then unbalanced parentheses i think that has yes that has read and consumed that parenthesis so at this point we're good so at this point we have read this bit of an expression we may next see another parenthesized register or we might not see another parenthesized register and this will depend on this will make the difference between whether this is this form an indexed register operation or this form which is a direct memory reference so we need to read the next token if the token is a open parenthesis then we must be seeing a parenthesized register so let's read the register we expect this value to be a register there are no other valid forms we now expect to see a closed parenthesis oh we do have an expect good so this if we saw a open parenthesis there then this must be indexed otherwise it's a direct memory reference yikes but life gets even more complicated because we also have these now we know that a training plus can only appear after a direct after a parenthesized register so that's here so we can actually put the put the logic in here so read the token if the token is a plus then otherwise undo the push uh compiles really nine two five nine nine five nine one yes the code size has gone up that compiles i mean it's not right but it compiles okay uh right what are our other choices we can have an optional at sign before any of this lot after an at sign we can have a parenthesized post increment parenthesized pre-increment indexed absolute relative looking at these stack relative instructions two six yeah these these are actually just these they don't even need to be described so we only have to care about these and these so in fact we can have an at in front of anything except a plain register now if the thing is a minus sign then this can be either actually i was thinking that this can either be one of these or one of these but a negative address doesn't actually make any sense so i think we can assume it's always going to be one of these so let us expect a open parenthesis and we can do a bit of simplification like this okay uh yeah this is pausing for you it was always a mess it'll be so much easier with an actual pauser generator and believe it or not i have one in cal goal uh ported to cal goal it's lemon but uh it's kind of overkill for an assembler and integrating it will be more trouble than it's worth so if it's a open parenthesis read the thing if it's a register i actually need to do that this is a parenthesized register which may be followed by a plus sign if it's not a register then this is wrong if it's not a yeah if it's not a register then push it back read the expression and then look to see if it's indexed or a direct memory reference what else can we have we can have a hash sign this is pretty straightforward we know the result must be a constant value we haven't done simple registers so we've read the token and it's not what we expected but it's not one of these rather so it must if it's a if it is a register then it's a register do not yeah if it's not a register then it's got to be one of these and you notice that there is no post incremental decrement form if you're using an uh an index which is nice so read the expression the result must uh or it could be one of these so if the result is a open parenthesis yep that's all as above we could probably factor out into an factor out a bit but let's not worry about that for now oh dear uh i think we have them all now i mean there's no actual logic here let's try running it and seeing what happens two registers registers are not instructions right that's because i haven't actually defined a ad instruction so that's turned into a label and then it's tried to execute this as the instruction so what instructions do we actually have um of simple register that's what we expected let's try to blur simple register it is indeed what we expected good that suggests that that is kind of almost working sort of now we actually have to do something with the results so we want to define some values for addressing modes and some variables to put the result of the read-in traditionally with this stuff we're just using global variables so what have we got it actually occurs to me that at rn is equivalent to this that makes the addressing modes much more consistent means that the bottom bit of the dressing mode indicates whether there's a additional dereference or not so a simple register post increment pre decrement index and a value that we or in to indicate that it's deferred and in fact we're going to do all this in octal because we want the register to go in the bottom octal digit so the values that we need to store for this are the addressing mode if any no in fact there will always be an addressing mode and the extension word if any there will always be an extension word right there will only be an extension word if we're using one of the indexed addressing modes which includes relative here or we are using one of the synthetic immediate addressing modes which actually produce a auto increment addressing mode with a data word that's not part of the instruction so it's in fact not going to be easy to detect whether we need an extension word from the from the addressing mode alone so let's use a third octal digit to express that like this okay so this means here when we will expect a register this is this is a always going to be pre decrement with a register that's going to be pre-deck ord with token symbol dot value u n 16 exactly when u n 8 was expected and no extension word we have an immediate we have a immediate this is always going to be and the program counter is always going to be register 7 where does uh where does read expression put the results token number token number so post increment register we have no extension word post ink read token here will overwrite the value therefore we need to save the register is token symbol value as 8 like so in this case a simple parenthesized register is going to be no post ink this is this yep register deferred indexed form right ah we have an expression that's a value which we need to save because the next time we actually read anything then it will be overwritten token number this is indexed that's going to be one of these direct memory reference right now this is going to be a little bit complicated because direct memory references are should actually be assembled as relative offsets from the current program counter the current program counter is of course not where we currently are the current program counter is the address of the extension word that the offset is going to be stored in so the way this is going to work is something like this we have the instruction we have the first operand and here we're going to have pc minus label of course when this instruction is executed then that's actually going to look like this followed by a data word so but when this is read pc we'll be pointing here and at this point the point to where we are actually parsing the operand we do not know whether it's the first or second extension word therefore we don't know how much well at the point where we read the operand the program counter is actually pointing here because we haven't omitted anything yet so we're going to have to put in a offset of either two or four bytes depending on whether it's the first or second extension word but we don't know which one this is yet so that's actually going to have to happen later now one way around this is to simply not generate position independent code and instead of emitting a relative emitting one of these actually one of these then we emit one of these like that this will look exactly the same except that here we actually put the address of the label it will work in exactly the same way but take a bit longer because there's this extra dereference needs to happen but I think I would rather not now uh how many spare bits do we have here uh I'm trying to remember what 255 is in octal 377 377 right we've got one more so what we're actually going to do is to set this bit that tells the actual instruction callback the things that's emitting the the results that this value is a relative thing it needs to be adjusted for the the program counter so where were we direct memory reference nine on a second that needs to go there so this is a relative extension post ink program counter do I have a you do have a program counter value program counter minus uh value minus program counter okay it does occur to me that the pdp11 is a segmented architecture where data and instructions are read from different address spaces I think I don't think this is something that the assembler has to worry about to be honest or at least I'm not gonna worry about it okay what's this one simple register that should be straightforward as you know equals am reg or to be in date indexed form and this needs an extension and this is a direct memory reference again right well 05 missing colon 722 missing colon you can tell I've been writing in cc++ and java recently right well we have a thing what happens when you actually run this expected a single operand yep put that back to mauve pad separator interesting I know what's wrong yeah okay read operand here needs to return the next token so when we have a read expression or something like that then we already read the token when we haven't expect we haven't so we need to put one of these in there token is read token if it's a plus I think that's right that's separator why we're getting bad separator so it should have passed through uh here twice ah we need to read another token here good that's work we have successfully assembled our first file except we haven't actually omitted anything we just have read two operands so what do we have we have put some debug information x i 16 this should be the first operand this should be the second operand um yep that's correct that has produced operands of zero one which is register number one so if i change this to pc you may notice a certain similarity to the arm here where the program counter is an ordinary register and you'll see more of that later we in fact get what look like fairly sensible results let's try that might be in trouble now that seems to have worked i'm just thinking that my syntax highlighting thinks this is a comment but actually that is a comment in this syntax so five seven should be five auto auto decrement deferred uh five seven wait a minute this is hex uh i never did print octal oh dear five seven hex is one two seven so one means has an extension word which is correct two seven is auto increment program counter which is correct okay we are getting somewhere i think we're almost ready to actually start emitting stuff so here we need to save the old the parameter one addressing mode because the next expect operand is going to overwrite it and at this point we are now ready to start actually emitting the uh opcode so so we use the opcode template that's stored in the symbol table we now want the source addressing mode but just the bottom two-bit digits of it shifted left by two octal digits which is six bits followed by the destination parameter addressing mode now if we have a parameter one extension word emit it if we have a parameter two extension word emit it you in 16 okay now we should have actually emitted something what's our output file name output okay we have something so let us disassemble it and see what comes out marv 2322 comma r7 um i think that's right you know one two three four and this is decimal i've just had a horrible thought that might going to have to change all my uh immediate values into octal i think if i want strict compatibility i do need to but i think that has actually worked and let's try that missing expression oh uh i haven't done um i haven't actually implemented this for one cb that's easy enough i haven't done any of the relative adjustments yet that's still not going to work but now i can debug why read operate so you see that's gone through here right i need to push the thing back onto okay and that has now that's interesting so that's actually emitted a immediate value which is not what i expected what that should have done was emitted a relative an incorrect relative value fact it is incorrect that should be a a octal 10 this is address eight yeah this is my data word so why is that produced the wrong result two two seven wait a minute two seven is auto increment program counter ah right the disassembler is trying to be clever it's actually read this relative value yeah oh what it's done is it's decoded the synthetic addressing mode rather than showing me what's actually there i wonder if there's a way to to stop that probably there isn't it's not something anyone would really want no okay no matter at least it's showing that i've got the right result then fact the fact i've got the right result is a little bit concerning because that shouldn't be right because i haven't done any of the relative uh adjustments yet but it is at least showing that we're getting there normally whenever you do a program counter relative addressing mode the program counter is not quite where you expect it is because of processor pipelining it looks like in this case the program counter is pointing at the beginning of this word rather than the beginning of this word that suggests that it is only incremented after the instruction is evaluated no wait a minute that's rubbish no this this is the wrong addressing mode this should be deferred no it shouldn't be deferred huh okay let's take a look at this again 2 7 is the right addressing mode relative should be 6 7 indexed but relative is not hang on a second so if you use one of these relative addressing modes then it's actually the the index addressing mode which does not adjust the register so the auxiliary prc for next instruction right this actually suggests that if the destination instruction is the program counter then you always get automatic incrementing even if you're not using the right addressing mode that's unexpected i don't see anything in here that explicitly says that though immediate and absolute are merely auto increment auto increment deferred applied to pc yeah i think that's this bit here it automatically increments it past auxiliary word right so this should be indexed rather than posting and the same here let's try that to see what happens and we should now get the wrong result okay let's try that uh the relative stuff so we need the addressing mode which is a unit 8 and the extension word which is a 16 so if there is no extension word give up if the if it's a relative value the program counter here is wrong so let's remove those okay it is an incorrect value so probably we just need a adjustment there that is now pointing at the right value so let's just do something a bit better one that's not right those should be pointing at different words image 16 does and adjust the program counter doesn't it yes it does hmm oh wait no i'm looking at the wrong fields yes this is correct move six to ten this is at address six this is at address 10 both octal it just so happens they're both encoded as the same value because uh the two offset uh note that you are allowed to do this although the result doesn't really make any sense let's just see what that does yep that's correct good right i am i think that works well it's not exactly tested but i can hope let's insert the other instructions so we want the double operand instructions which are these the pzp11 is way more orthogonal than the 8080 so there are in fact fewer things i need to hard code so bit bit b bit bit b bis bis b add sub bit bit b uh bick and bis are the rough equivalents of or an and bick is uh and with an inverted value bis is or this means that actually getting and is you have to negate source uh i'm i've seen this before on other architectures so oh three one three oh four one four five one five oh six one six right these are some exceptions used for special instructions these will probably need their own core back so i won't fiddle with them for now includes uh things like multiplication which these got wedged in later as an extension so they don't really fit the encoding scheme but let's do these so jsnemt i done three is that right yeah i'm doing these in the wrong order okay so let's just do a bunch of those jsnemt are special so i'm not going to touch them for now it's still using one on the top bit to indicate that this is a byte sized operand rather than a word sized operand actually it can be cleverer than this uh clear com compliments this yeah this is the negation i was mentioning yes mentioning earlier ink deck neg dc just be raw b roll b s i s r b these shift by one bit but there are some extensions that allow um shifting by multiple bits that got added later the pdp11 went through multiple versions of the architecture okay and now we get to the that should be mark and one two three four five six seven b mtps mfp oops mfpi fp fpd yeah i'm getting tired of my typing is suffering tpi tpd sign extend move from process of status so 1067 1067 1066 1066 1065 1065 1064 um i've got myself muddled there's no asrb that's why so 1063063 10 hang on i've got asr in twice that's the problem aslb is 1063 0063 one oh six two oh six two another two asr's seem to have a lot of asr's in that roll b is 1061 1061 1060 1060 test b1057 1057 1056 1056 edcb 1055 1055 neg b1054 1054 deck b1053 ink b5 2 comm 5150 and swab is 003 okay notice that the adc and spc instructions don't do what you might expect from others from other architectures rather than being a two operator add plus carry these just do the carry adjustment for an earlier addition done with the two operand add or sub these two right and so the conditional branch instructions are special because they have the embedded offset and this is going to bring us to another interesting feature of this this architecture that this has in common with many other architectures it's something the assembler is going to need to deal with but actually i think i'm going to do jsr and emt first jsr is 004 let's just line these up nicely i don't need these anymore right jsr so you remember how i said that this was a little bit like arm the way jsr works is that you give it a register and you give it a label and let me actually just double check that i'm right about this yep what it does is it pushes the value of the register onto the stack and then later when you return from your uh subroutine use this and it pops the value of the stack into this register now if you're just using r3 this doesn't really mean much but normally you'd use pc so this pushes the old value of the program counter onto the stack and this pops the value on the stack into the program counter now this does actually suggest that rts is encoded as a simple mob because you can just no i'm sorry i'm completely wrong about that it pushes the yeah i looked this up earlier but i seem to have got it muddled the old value of r4 will be pushed to the stack and the return address would be an r4 ah right and if you use pc it in fact does not you not do this yeah this is used for varag calls where one of the parameters wants to be the stack address so yes you would use r3 you would uh old value of r4 and the return address would be an r4 oh yes right this is so that this is to allow you to access values immediately after the jsr so that for example you could do so that when the jsr is called print s gets uh r0 points at the string which is immediately after the jsr statement it can then increment r0 and then it finishes with ret r0 and this value goes back into the program counter but if this is a pc then this is a special form that doesn't do any of this okay that's not the way i thought it worked but anyway what we have to do here is read an operand well in fact the in fact it works like this and then we read the second operand and this is in yeah there's the second operand is the destination and it is encoded in exactly the same way used for an op1 but the register is is encoded into instruction um yes that for is part of the opcode and not part of the addressing mode so if i go look at it it's a it's an auto decrement yeah that makes no sense as a addressing mode param 1 addressing mode not initializer of wrong type and then symbol fall back yeah that was fiddler than i was expecting okay so let's put in pc comma data 2 first parameter must be a register first parameter is a register it also occurs to me that i completely forgot to do this so we should go through oh it's got the register in it yep if addressing mode 370 that masks off the bottom digit which has got the register in it there we go and we are calling the 2 here that is the right value yes and you notice that this has the disassembler here is omitted to the pc so we should probably allow that form in the assembler but i'm not going to worry about it for now and rts is very similar what is the the encoding for rts is not in this table yep okay i'm going to have to go and find the real documentation and i could probably get a t refill as well so back in a moment okay here's the 1979 pdf of the uh the processor handbook and the rts instruction which i just had a moment to go before i moved away from it here it is uh has got the register in the bottom three bits and i has got opcode 2 0 so oh 2 0 rts cb so this is going to be very nearly the same code except only with one operand like so three or not let me double check this ah 0002 0 0002 there we go right yes rts pc comes out as red ah and jsr pc comes out as call right that's actually i can implement those very easily so let's actually just do that so a simple cb instruction has got takes no parameters and it just emits that value i have no call cb so all we need is a simple operand like so and we have no simple cb but we can put one in very easily that simple jsr r3 data 2 call data 2 jsr r3 32 jsr that's the wrong operand again there's seven there for the program counter there we go nope did that wrong that should be a zero there on a seven there that's better jsr pc tends to call jsr r3 is a jsr r3 call is a call rts pc is a ret rts r3 is an rts r3 and a ret is a ret excellent right so um let's look at the next group of instructions yes conditional branch instructions right now because these instructions have a limited offset if you try to branch further than can be represented in one of these instructions then we either have to fail with an error or generate different machine code now cal goal is a compiler so it doesn't really know how big code is it's not its job therefore it really wants the assembler to automatically expand any of these instructions now the way this works is let's remove some of this nonsense say we have how does this work bne yeah these these work based on the processor flags so they only take one parameter so normally we get this which becomes something like that where the two three is the encoding of the delta if the delta is too big then we have to change the code to produce this so where this is a single 16 bit instruction this is now three 16 bit instructions we've got one for the beq um i think that's a four followed by a jump this means that instruction changes size depending on the code layout we have a two pass assembler so that uh this is to allow exactly this sort of thing uh so we can make this work but we're going to have to be really careful it is possible to produce bad code by doing this wrong now let me take a look at these instructions uh vr is branch always it's kind of special i'm not going to i'm going to ignore this one for now i'll have to implement that especially but all these other instructions are in pairs yeah which are negated good they're all just checking the logic is correct this means that we can negate the instruction by flipping this bit which is nice that makes the implementation much easier okay let's add them both four like so and we have one two three four five six seven eight nine ten eleven twelve thirteen fourteen fifteen entries so we've got bne bq bge blt b gt ble bpl bmi b high b loss bvc bvs bcc bcs bhis below and these last pair are just aliases for each other and this is going to be four zero four zero four zero all the way down two two three three zero one one two two three three three three okay so the reason why there are so many is we have signed and unsigned comparisons so these will all be unsigned these check for non negative or negative these check these do signed comparisons yeah the uh i need to figure out the difference in b high and b his higher than oh higher or same okay yeah uh actually looking at this yeah these instructions are signed the xor is a giveaway well these ones are unsigned and we have two plane equality instructions okay let's go and implement con cb so we are a two pass compiler there's two there's two ways of doing this which is to be be a two pass compiler or a multi pass compiler a multi pass compiler generates better code a two pass compiler is simpler so we're actually going to do this in a two pass way the the issue is that as your code changes size your various labels might move in and out of range of short encodings and if you're not very careful you can end up with a situation where the assembler will decide on one set of encodings uh move on to the next pass decide on a different set of encodings and either generate bad code or reach a situation where it can't actually uh some of the labels have moved out of range again and if you're not careful you can end up with infinite loops where the compiler continuously rearranges the code and labels move in and out of scope and it just doesn't work so for a simple two pass compiler what we have to do is do pass one based on every instruction taking the maximum possible size which is three words six bytes and then in pass two we shorten any instructions that can be made shorter it's not as efficient as a multi pass system because it's possible that in pass two now the instructions are shorter we'll be able to encode some things using the short version but it is simple so now the result is always going to be a label so I don't think we want an operand I think we want a expression so we read the expression if we are in pass one then don't do anything except increase the program counter if we're in pass two then we actually omit the code so let's figure out the delta token number token number and the offset is let me check this the low order byte is an offset relative to the current location this offset is a number of words and it's signed okay so make sure that interpreted as signed this can vary from minus 256 to 254 if it's within this range then we can use the short form and we want to do shift it right by one and truncate it okay if it will not fit then we need to change the sense of the instruction use an offset of four I believe we'll have to check this and then emit a simple jump and our jump is that's a single op instruction oh I haven't implemented that yet oh wait a minute jump is uh jump is moving a value to pc so it's just a mauve let me check to make sure that is touch only to second word control blah blah blah opcode 001 dd oh d dd this is not a mauve instruction um hmm I do wonder why it's not a jump instruction uh not a mauve instruction so the dd at the bottom of this is the uh addressing mode of the theme to jump to I am surprised but let's stick with what it says here just double check this is all this is all this has on jump no actually useful information okay so 001 because we want a immediate value this is going to be in fact it's going to be one of these relative extension em index by pc so index is six that's six six token number minus program counter plus two I think unit 16 and unit eight are not compatible our cal goal requires you to be completely explicit about conversions so uh when we do this shift we actually want it to be an unsigned 16 bit value because that shifting those is cheaper than shifting signed values and the result has to be a uh 16 bit value before it can be whoops it should be an or before we can or it in with the value right well that compiles so let's try so beq data one cluru data two label already the find you think think I have a problem so you get this error if you try to define a label to a value that it hasn't been defined two before and this is happening because the first pass through the code this is six bytes and this is two bytes thus giving this a value of eight sorry this is four bytes giving a value of 10 the second pass through the code this is two bytes this is four bytes thus giving this the value of six that's what it's telling me here 10 versus six now that is actually legal no that's not legal right I am going to have to change my framework this needs to be a three pass compiler so the thing is the the first pass through we get these values the second pass through we get these values but uh if we refer to data one up here somewhere then in the second pass when we emit this we're actually emitting the wrong value because it has the value defined in pass one when it's at 10 but it's not anymore it's at six well you only know it's six at the end of pass two so we're actually going to need a third pass where the values are the same as at the end of pass two for actually emitting the code now we didn't need any of this with the 8080 because the 8080 didn't have instructions that change size so we're going to have to change the framework that should be easy enough the framework's just this bit so at the end of pass one all value all symbols should be defined then we do pass two to shrink the code then we do pass three to emit the code so we only want to emit and pass three we only want to print the title and pass one okay that's reasonable uh this needs to be maximally expanded in pass one but the right value in pass two and three we only want to emit the listing in pass three only make the listing in pass three we don't have to fix the listings i bet that's wrong as well okay now that's still not going to work because we get the label already defined error now i think that you are allowed to redefine labels in passes two and three so if we're in pass one and the value of the label is changing and it's undefined yes that should work and we get a invalid opcode yay we get several invalid opcodes oh that the first invalid opcode is this dw the second one is the beq which is just hopelessly wrong so let's check here what have we got okay that's interesting okay so we now have the correct bne except that should be a beq okay did we get these right bne should be 1400 that's better so bq 10 data one is at 10 it's here okay so let's put lots of stuff here i think i need more stuff in there wait a minute it's not done anything like what i expected yes cb oh that's a that's a nasty bug okay what ds does is it uh just defines zeros in the output file this is actually not emitting any code it's just increasing the program counter as this was copied from the 8080 then that's kind of the same problem this will have worked fine if ds is at the end of the program okay so this is going to need to be i'll take the number as zero loop okay i didn't insert enough stuff so we can still actually reach the instruction but at least that's now doing the right thing that's all wrong it's using the wrong register and the wrong address mode so for a start this offset here is wrong it's off by four four really oh yes uh yeah there's the offset is a number of words rather than a number of bytes so it needs to be shifted right by one that's better so now this bne is pointing at the right place this clear instruction the jump instruction is trying to get to 640 is that the right address no it's off by four i think i need a minus two there so this is now jumping to 634 right that's correct so there's conned working br is similar the difference is that there is no actual condition involved conditional involved so the largest value the largest size it can be as four bytes and if it doesn't fit then we don't want to omit the jump over so it should just be so we have jump 632 which is the right address while if it's shorter it's a branch to 10 which is the right address good right we have conditionals working uh and actually just before i do anything else as an arch 8080 dscb let me just stick that in the 8080 assembler leave that with that file okay right where else have we got we've actually done the bulk of the instructions we have some one and branch oh yeah right these are the special one op instructions quiet register source operand yeah so this is the same as a two op instruction except the the opcode extends into the three bits of the source instruction the source operand that normally represent the addressing mode so you just have the uh do the register bits sob here is going to be quite special because it's another branch instruction okay now let's take a look at these i think these are all the same i may have to look these up in the real documentation yeah these these are actually instruction extensions unfortunately that doesn't appear to be a nice table of uh oh this is the floating point unit that doesn't appear to be a nice table of instruction encodings which is a shame that's one reason why i wanted the wikipedia article because it looks simple div divide instruction 75 yeah okay ash and ash c here it is the number of bits specified in the count field bits five to zero of the source operand right uh the source operand therefore is not a era it's not an addressing mode it's a constant yeah okay these are these are kind of weird let's do mull and div verify they work these seem to be similar which is nice this actually does seem to be the encoding table but it'd be nice if it was a bit denser try div divide 07 1 i think mull is 070 so this is a two op two operand instruction did i actually implement expect register i don't think i did i didn't easy enough to fit and of course there's no extension word for the source there is an extension word for the destination yeah that should be it don't need anything else so mull r1 by data one div r2 by r3 yeah that's not right let's just get rid of this as well uh the r1 r and rv1 is divided by the source operand the looking at the disassembly here uh the the order of the operands is source destination so that looks like the destination is the thing on the left so in fact these need to be the other way round so the encoding is inverted based on uh that's really not right the encoding is okay div is correct but the encoding is inverted relative to the other uh instructions right because the source is the one that's got the extension word we do need to copy it here and it's the second parameter that needs to be a register well that's not right either 7707 okay okay that's better 14 yes with r1 r2 and r3 okay good all right what's next ash and ash c actually i think these might be the same encoding let's just give that a try and see what happens i have to rename the uh have to rename the helper tool yep that's fine that works why is there an extension word that's because this is not a constant this is whatever's at the address in nine so if you want a constant we have to do this yep okay and remember that this is octal so 11 here is eight plus one which is nine all right let's rename this ash c is seven three xor is seven four that should be a two look at symbol during in it xor this suggests that there's already a xor oh go blast uh yeah the instruction is conflicting with a operand uh i assume we're actually doing something with that operand is not just dead code push and apply operator yeah i have to rename one of these it's going to be this that's a pain anyway that does seem to be working so we're at s o b now i'm going to ignore floating point instructions until we have an application that needs it because calgo doesn't system instructions i'm going to have to look up right where's s o b because this is going to be weird subtract one and branch if not equal to zero octo 0 7 7 r o o plus six bit offset yeah that's exactly what i expected actually the offset goes in the bottom two bits this is wrong uh i'm so used to hexadecimal that they said two digits and i immediately thought 256 but that's actually going to be plus or minus 64 words that's 128 is that right and 64 words 128 by yeah yeah that'll be 128 126 and that gets all into the bottom two bits there so uh our sob instruction is actually going to be very similar to cond sob is 77 we expect a register yeah a simple expect register is not going to help there because we still need to deal with situations where it's the first or the second parameter so let's just keep doing it longhand followed by an expression and we want to save the parameter one addressing mode now sob's tricky the way it works is it essentially does and branch if not equal to zero like this so in order to in order to expand this that is if desk is too big then because we cannot invert the sense of this we're actually going to have to do this yuck yuck so that's one two three four words yeah i don't think there's any other option there oh what if we do it longhand so that would be deck r4 so sob is equivalent to this i assume it's equivalent to this does oh sob doesn't change the condition codes so replacing it with something that does like uh yeah like deck means that in the long form it will corrupt the condition codes when we're not expecting it the uh does that he suffers from the same problem with dj nz so no i'm going to have to do this long form blast okay so program eight picks this as well seven it's two octal digits not two hex digits so we're actually going to jump forwards two bytes i think that's actually yeah two bytes the branch is going to jump forward four bytes so this is the sob this is the branch which is 0004 0004 uh two i think then we get the jump to the destination that delta is rubbish two that should be correct actually it's not let me double check the docs the offset is a six bit positive number the sob instruction cannot be used to transfer control in the forward direction oh oh you okay that's not at all as i was expecting that means this is a also i think these numbers i think this delta is still wrong yeah so that's minus 256 and delta is less than zero so this branch is backwards okay two times offset so let's divide by one okay well let's put some code in here so we can actually make it jump backwards right that is actually doing the right thing we cannot use sob to jump forwards actually we can't use it here this means this isn't going to work what we have to do is so okay that should work yeah um i'm slightly struggling to see how useful this is so this is the branch this is the jump and then last is the sob instruction which goes back for so what do we have br 36 which goes to here which is the sob instruction sob 34 that's the wrong result this needs to be four really thought that would it's a number of words yeah that should be a three because we're including the here we go sob r4 to r32 which is here which then jumps to 46 which is the two that's the correct result right that is fairly loathsome uh it's also possible for us to shrink that jump down because if we're jumping forwards a small amount so else if so the branch you we lose one word this now becomes a branch i think that will work unexpected so br 34 this is wrong that wants to be a that wants to be a that wants to be a three that wants to be a two br 34 goes from here to here uh if the sob is correct it goes from here to here and we do the br to 46 and if you put lots of stuff in there then we have br from here to here sob from here to here jump to 666 which is a two wow okay uh yeah thinking about compiler generation this means that we should only really generate a sob at the end of loops so it jumps backwards i am not sure whether calgirl's good at this because the front end compiler doesn't really know about the different it doesn't know about what all the different loops are yeah okay well we haven't done emt which is one oh four oh oh we're actually getting there so only a few classes to go which is good because i have to be done in about an hour and a half right the the bottom byte can be anything so expect expression commit 16 so the value if token number is greater the token number unsigned yes it is okay that should be easy so what is 52 in octal i have my trusty hp 48 here so let me just type five two five two equal based real is 42 yeah that's the right value good where are we now done all these hopefully done all these done not done jump yep not done jump yeah this is the one that i was wondering why it wasn't a morph to pc that's oh oh oh oh four value or addressing mode in extension mode extension word ump data two and just because i'm curious data two pc well that's produced garbage that's the wrong opcode yeah so i think these both do the same thing interesting probably jump is faster there's anyway five six rti that is the right value yep that's jump mark what's mark use as part of the sander pdp 11 subroutine return convention mark facilitates the stack cleanup procedures involved in subroutine exit assembler format is mark n okay that's looks easy enough six four this is very similar to omit cb except with a different range you look at symbol during in it mark okay that's right mark seven is just what i said emt trap and buput presumably breakpoint i have a feeling these will be fairly simple condition code instructions we need to do those in a moment what's this that's a very poorly scanned picture of some pdp 11 users all i can see are suits traps and emt's are identical in operation except they use different addresses okay so where did we put emt wrap three that works bpt breakpoint trap that is a simple instruction i also remember that halt was there too so that's easy to do yep and halt is this okay what's next iot input output trap yeah we're getting into the pretty woolly ones now actually these are zero one and five so let's just stick those in zero one iot should be here somewhere four i wonder what instruction two is rti and rtt rti is instruction two rtt is six okay condition code operations uh i symbols scc and cc assembly instruction set or cleared respectively all four condition codes yeah i don't think wikipedia is helping here so we are set selected set condition code bits selectable combination of these bits when we set together a condition code bits corresponding to so this actually looks interestingly complicated because you can set or reset any combination of the condition code flags what it doesn't do is go into much information about the syntax so the base is two six zero so how comes the hp 48 binary to real that is one seven six so let's try this and see what comes out so the disassembler says they're just invalid op codes set the bits specified by zero one two or three so this looks like this disassembler only understands the instructions for setting single bits and if you want to do anything else you roll your own that means it's pretty easy to implement so we've got sec set all which is 000277 and we've got sec which is 000261 v two z four c seven oh and there'll also be the equivalent c instructions should be here somewhere here we go which are exactly the same except they're based on 240 rather than 260 oh and these are cl rather than ccc clc okay so let's put them in our test file the n and cv z ccc clm clc clz and cv z and cv z and cv z okay well i think i am probably done with most of the wikipedia instructions so let's go to here and scroll up to the top of the instruction list and there's work through them adc adc be done add done ash ash see done done done i'm doing this from memory professionalism i've heard of it blt bmi bne blah blah blah brc com we did do com yeah csm call to supervisor mode copies the current stack point to the supervisor mode and the argument word addressed by the operand says well well it's a one op instruction 000701cv csm well apparently that is not correct so that is a destination register ah that means i can't use one of these so let's just do one of those that's still wrong maybe this disassembler doesn't support it it does say it's only on the 1144 oh well i'll leave it in deck div e n t halt ink iot they do iot i did do iot jump yep j s r ooh ldub cause of the lower eight bits of general register three to be loaded into the micro break register if you say so i wonder if that's supported you know yeah i think this disassembler only supports more recent stuff well let's just go put these in anyway uh med maintenance examine depth processor specific maintenance function first word is use an escape of the code representing the operation and the address okay wow there's a lot of it uh mfpd moved from previous data space we've done these they're up here mns maintenance normalization shift this is a floating point instruction therefore i will ignore it mov mov b mpp maintenance partial product use for diagnostic purposes to test the multiplication network this looks floating point so i'll ignore it done those done those done those done those raw rti rts ut that did do it yes it's there sbc all the s's sob spls at the priority level right this is another special 0002 3 and it takes a small expression right that one it understands uh subs swabs six six trap test weight xfc user control store utilizes the xfc instructions associated with the ucs option i'll just ignore that one xor okay well that seems to be the lot so that would be it done how am i going to actually test this to any degree well we've got the disassembly of the oh i haven't tested any of the the at sign instructions let's put a couple of those in see what happens okay that worked um i don't have any source code i can use this disassembly of the the pdp11 assembler but i would have to un octal quite a lot of it uh yeah remember what i was saying about jsr when i got muddled earlier this is doing exactly it so this is calling the routine at 166 r5 is pointing at this piece of data here which you can see is not data so if you go up to 166 there's a routine here which does stuff and it eventually should end with a ret r5 rtsr5 here we are uh yes i was as i was saying i would need to un octal five some of this so i think i will just see this looks like a likely piece let's just do this chop out this there's a semi colon so we can leave them registers are not instructions at 9 13 oh how did i miss kump did i do test as well there's test no kump yeah kump very important uh compares instruction compares two values you call kump and then it should be a two opcode instruction o2 ssdd it doesn't subtract but doesn't actually write the result back to the register so why isn't that in the it is here i just missed it kump kump b so you call kump and then you call one of the conditional branches error at line 15 expected value got operator okay that's probably finding this that will be a flaw in the expression evaluator yeah so it sees the closed parenthesis and it tries to uh terminate the um it tries to find the matching parenthesis but there isn't one um i think i need to write this in value equals zero turn so it now when it sees a that's not gonna work uh expected value got operator right it's actually seen want value think i don't know what this is doing uh okay let's try to put some debugging in okay uh so token 254 is 255 254 it's a number then it's seen a 40 which is a at sign oh hang on this is that this is a decimal not hex so 40 is a open parenthesis right right what's happened is it's read the two and it's looked at the next character and it's expecting it to be an operator or a known terminator character but an open parenthesis is not a known terminator character so it's hit this now so if seen value is not equals to zero then it's expecting a expected value got operator that does seem to be the wrong message because i would expect it to be wanting a operator but yeah that was what made it work so this we've assembled and disassembled the thing and this looks more or less right the numbers are all different because these are being interpreted as a decimal clear blah blah blah be any 1474 it's an expanded that because 1474 is out of range to our six clear be call the thing move a thing likewise another expanded branch add swap decrement expanded branch okay that looks promising i think i have a thing that works so we now have a working three pass compiler which is nice it's well on my machine here it is a whole 12k of code it makes a 18 kilobyte x86 binary that should have built it yeah that's built it for all the other cal goal architectures so you can now assemble pdp 11 machine code on a bbc micro on cpm on z80 cpm 12 fish k more once you apply the rounding although we don't have the pdp 11 compiler actually working so i can't show you what this is on a pdp 11 not that i have a pdp 11 or the three phase power supply but yeah i think that is now done i'm sure there are bugs 11 and push it so i can't pretend i didn't do it and it's done well thank you very much anyone who stayed with me through this i hope this will actually be useful to me and possibly even to other people it's good that i now have a i've sorted out a lot of the problems involved in turning this into a three pass compiler with expandable instructions and addressing modes it'd be good to attack the z80 at some point it would be also nice to do like x86 assembly but that's just so grim what makes it worse is that if you look at the encoding there is actually an underlying logic that dates back to the 8088 but it's been just so mangled as they added more and more instructions into the corners it would also be good to factor out the stuff in common between this and the 8080 version there's actually more than i thought like all the expression parsing and general tokenization so that would make maintenance easier but i'm gonna do all that offline anyway thank you for watching i hope you enjoyed this video please let me know what you think in the comments