the usual way to get a large computerapplication developed involves a bigteam of people working in closecoordination most of the time this workssurprisingly well but it does have itsproblems and large projects tend to getdone poorly they take a long time theyconsume an astonishing amount of moneyand in many cases the individual teammembers are dissatisfied so everybody inthe computing business is constantlysearching for ways to do a better job ofdeveloping computer applications therearen't likely to be any final answersboth because the problems are hard andbecause as we find solutions we try evenmore ambitious objectives but there aresome things that can be done to makelife easier for everybody on a largeprogramming project a good programmingenvironment helps a lot and in the nextfew minutes we're going to show you someof the properties of the UNIX operatingsystem that make it a good programmingenvironment for many purposes in BellLabs as in many industries almosteveryone has some kind of involvementwith software either they are actuallyproducing software and that is their jobor they are impacted by software or theyuse software fact that Bell Labs about50% of the people are actually producingsoftware and everyone else has some kindof involvement with itin fact that's one of our worse problemstoday there is a crying need for usefulsoftware to do effective jobs we just donot have enough people for write allthat software keeping large amounts ofsoftware working and keeping it workingin the face of change is a big job takesa lot of skilled people to do this nowsoftware is different from hardware whenyou build hardware and send it outyou may have to fix it because it breaksbut you don't demand for example thatyour radio suddenly turn into atelevision and you don't demand that apiece of hardware suddenly do acompletely different function but peopledo that of software allthe time there's a continual demand forchanges enhancements new features thatpeople find necessary once they get usedto a system in other words we put thesystem out there people get used to ittheir jobs change they come back withmore demands for different sorts offeatures in the system the result isthere's no way to get perfectrequirements in the first place and thatmeans that we have to build the softwareto be very changed tolerant because wedo not want to throw the software awaythe year after we wrote it there are acouple ways to do that one is to makethe software fairly clear and easy toread and understand and change and youdo that with some of the current popularstructured programming techniquesanother way is to write many many smallmodules of code that way when you have achange perhaps you only throw out a fewsmall modules or make changes in a fewmodules rather than in thousands andthousands of lines of code what weshould be doing in the computingbusiness is trying to raise the level atwhich we work so that a programmer canwrite a few lines of code that turn intomany many instructions in the machinethat way when changes need to be madeone just changes a few lines of coderather than thousands and thousands ofthem in the mythical man-monthfred brooks estimates that it took 5,000staff years of effort to produce theoperating system for IBM's 360 seriescomputers clearly nobody is going to dothat sort of thing very often certainlynot for every new type of hardware orfor every new class of applicationsomeone once said that software standsbetween the user and the machine and tome this conveys this picture of a greatwall of software up there that you haveto overcome to get anything done there'scertainly a grain of truth in the remarkanyway if you stop to look many manyoperating systems seem to spend asubstantial fraction of their time andeffort not in helping you but inimpeding you in making your jobdifficult sort of providing obstacles tobe overcome when Ken Thompson and DennisRitchie startthe unit system 1969they found a structure which simplifiedmany aspects of the interactions betweencomputers and people Thomson and Richiewere aiming to keep their system simpleand they found a collection ofprimitives that enable them to do agreat deal with the very few primitivesa UNIX system is made up sort of ofthree layers if you like the centrallayer the kernel is the thing thatcontrols the resources of the machinethen wrapped around that at least inconceptually is something called theshell which is the interface betweenmost users and the kernel part it sitsthere and waits for you to type commandsat it and then it interprets them andthen around that sort of an yet anotherlayer are useful programs things likeeditors and compilers for programminglanguages and document formattingprograms and programs that you writeyourself and what you can do is to thinkof these UNIX system programs basicallyas in some sense the building blockswith which you can create things and thething that distinguishes UNIX systemfrom any other system is the degree towhich those building blocks can be gluedtogether in a variety of different waysnot just obvious ways but in many casesvery unobvious ways to get differentjobs done the system is very flexible inthat respect I think the notion ofpipelining is the fundamentalcontribution if the system is you cantake a bunch of programs two or moreprograms and stick them together end toend so that the data simply flows fromthe one on the left to the one on theright and the system itself looks afterall the connections all of thesynchronization making sure that thedata goes from the one into the otherthe program's themselves don't knowanything about the connection as far asthey're concerned they're just talkingto the terminal let me give you anexample of how this works in practicethe system is as I mentioned is used alot for document preparation kinds ofthings programs for helping you typeletters or produce technical papers orwrite books in all of those things ofcourse people when they're typing intomachine make spelling mistakes so let'ssee how we could use some of thesebuilding block notions in practice tohelp you develop a program for findingspelling mistakes suppose I take asentence this is asentence which is in a paper that JohnMassey and I wrote some years ago now ifyou look at it carefully you'll see thatthere are actually a couple of spellingmistakes in there now suppose that wewanted to find the spelling mistakesusing a machine how would we do thatwell basically what we would do thesimplest thing I can think of is tosplit the sentence there in two wordsindividual words and then compare thewords against a dictionary and everytime we find a word which is in thatsentence but not in dictionary it's atleast a plausible contender for being aspelling mistake now how do we do thatwhat I want to show is that you can dothat using just existing UNIX programsjust gluing them together to get the jobdone suppose that we say first we willtake a program called make words andwe'll run that on the sentence now whatthat does is break the thing up into oneword per line now I'll take the outputand I'll pipe it into another programwhich will simply convert it intolowercase the reason I want to convertit into lowercase is that my dictionarydoesn't have any capitalizations and sowords like Bell and UNIX which arecapitalized here would show up asspelling mistakes unless I did this nowthe next thing that I want to do mydictionary is in fact sorted inalphabetical order dictionaries are andso it's a lot easier for me to comparethe words of my document to the words inthe dictionary if they're started so I'mgoing to run them into sort and finallyif you look at it carefully it doesn'tshow up very easily here but there arein fact there is a duplicate word theirsystems appears twice and in a realdocument words like TIG would show upmany times so we'd like to get rid ofduplicates so let's throw that throughanother program called unique so whatwe've got so far is we've got the wordsof my document in this case the sentenceone word per line in lowercase neatlysorted and all of the duplicate wordsthrown away so there's only one word oneinstance of each different word thenwhat I'm going to do is run it into onelast program called mismatch which willsimply print all of the words that camedown this pipeline andprint out the ones that were in thedocument that word in the dictionarywhat we have here is five separateprograms cooperating to do this job andin one giant pipeline now if you look atthe list that came out you'll see thatindeed we got laboratories and providewhich were our two spelling mistakes ofcourse we've got two other words as welland this tells you not only was whatwhat's good about the approach but alsowhat's bad about it time sharing is nota spelling mistake but it's a perfectlyfine example of technical jargon thesort of thing that means something toeverybody in the computer business itmeans nothing whatsoever to people whoare not computer types and the word UNIXis a fine example of something that'snot going to be found in a normaldictionary so what do we do first wetake the misspelled words and we go backto the original document and we correctthem so we don't have any spellingmistakes secondly we take the words thatlike time-sharing and unix that are notspelling mistakes but which showed uphere and we put them back into ourdictionary so that the next timesomebody has a document that containsthe UNIX or time-sharing they don't showup as spelling mistakes so we've notonly done our own job but we've improvedthe tool that we're using in the processso you notice that I did that whole jobwithout writing any programs at all thewhole thing is cobbled together out ofprograms that already exist it and all Idid was to use the fact that the systemprovides this mechanism of the pipelineso that I can take programs and stickthem together one after another to getmy job done and I think this is one ofthe reasons why the system so productivethat there's a large collection ofthings that people have already builtthat we use and as we build our newthings then they become part of therepertoire of things that peoplesubsequently can build on during thelast decade we have discovered a numberof new powerful pattern matchingalgorithms that are useful for locatingpatterns and texts many of thesealgorithms have been developed usinginsights obtained from theory obtainedby studying automata and language theoryas our knowledge of pattern matchingalgorithms increases we can very quicklytake this knowledge andpackage it in the form of UNIX programsand we can spread these UNIX programs tothe entire community very quicklythe UNIX systems has many features whichmake it easier for the programmer towrite programsthese include form atlas files thehierarchical directory structure theability to pipeline the output of onecommand as the input of another deviceindependent i/o all of these things makeprogramming considerably easier than onmost other systems the heart of thesystem is really the file system theability to store information forextended periods of time and the reasonone of the reasons the system works aswell as it does is that the file systemis well designed and many systems youhave to say an awful lot about a filebefore you can do anything with it youhave to say where it is and how big itis and what kind of information it'sgoing to that's going to be in itall kinds of things that are basicallyutterly completely irrelevant here youdon't have to do any of that the file isas big as it is it doesn't matter whereit is as long as you know what it'scalled and so you basically don't haveto think of any of those complexitiesthat you have in other systems when youwant information in a file you put itthere when you want it back you get itout again and you don't have to thinkabout size or number of Records ornumber of fields or anything like thatunless it's really germane to yourprogram for most purposes it's utterlyirrelevant a file is simply a sequenceof bytes its main attribute is its sizeby contrast and more conventionalsystems file has dozen or so attributesto specify or create a file it takesendless amounts of chitchat if you wanta UNIX system file you simply ask for afile and you can use it interchangeablywherever you want to file the UNIXsystem consists of a hierarchy ofdirectories which a directory is simplya file that contains the names of eitherother directories or files and thiswhole thing goes on recursively when youlog into a UNIX system you normally aresitting in a place that's called yourhome directory or users directory and Ican say TWD which means print the nameof my working directory and it'll tellme where I am it says at the moment thatI mean user bwk that's where I startwhen I log in now I can go up a level inthat I canchange to parent level and now if Iprint my working directory I'm in slashuser and I can go up one more level tothe root of the whole file system let mego back down to BW k and I can list thedirect the files that I have in thatdirectory and I find there among otherthings a directory called TV and I canlist the files that are there and I'llfind among other things the sentencethat we printed in a spelling mistakefinding program they look at that andsure enough there it is so as you cansee the file system hierarchy makes itpossible for users to organizeinformation into its natural groupingand to go up or down and find thingsquickly and easily the UNIX systeminterface for most people is throughprogram called the shell or the commandinterpreter basically it's simply aprogram that watches what you type andtreats it as requests to run particularprograms now there's nothing magic aboutrunning programs the programs that yourun are actually just the names of filesin the file system the shell searches inthe file system in a particular way tofind a file whose name is the name ofthe program that you think you'rerunning and it goes and executes it andin fact it's not possible for you as auser just by executing a program to tellhow that particular program has beenimplemented for example it might havebeen written in a language like Fortranor C or it may have in fact been writtenas something like the spell program thatwe talked about earlier which is acombination of other programs stucktogether with pipes or some similarthing and all put in a single file as ashell sequence or sequence of commandswhat Brian did earlier was he typed allthe commands the the five program namesfor his spelling checker on one lineusing the pipeline facilities that'snice except that you may want to checkdocuments often and you don't want tohave to type that long sequence ofcommands so it's possible to put all ofthese commands in a file and tell theshell whenI type the name of that file I want youto execute the commands that are insidethat file let me show you an example ofthis we have a program called splinewhich fits curves to a set of datapoints and I've got a set of five datapoints that we're going to see what thecurve looks like I'm going to run splinethrough a program that turns this intographics called graph and I'm going torun that through a special program thatturns the graphic language intosomething specific for this terminal Ionly need to type plot and data becauseinside the file plot is the string ofcommands and here is a result ofplotting those five data points on thisparticular terminal the ability to putcommands in files and only have to typethe file name to get these commandsexecuted makes the computing business alot easier often you're doing thingsthat are repetitive and you don't wantto have to type long lists of thingsthis makes our life much easier andallows us to tailor our environment forthe way we want to work another nicefeature of the UNIX programmingenvironment is the concept of inputoutput redirection normally when youtype a command the output from it goesto your terminal and the input comesfrom your keyboardhowever the shell can be told by asimple notation that when you run aprogram you wish the output to bedirected into a file or that the inputbe taken from a file for example toprint the output of my spelling programon the line printer instead of puttingit on my terminal all I have to do issay my spell sentence greater thandevice line printer rather than my spellsentence and the output goes into thefile what looks like a file except thatit's actually a file that causes theline printer to spring into action andprint my three or four spelling mistakeson the line printer on many systemsredirection of input and output isliterally impossible because theprograms have wired into them the notionthat they have to read or write theuser's terminal and there's simply nowayto convince them otherwise they'd haveto do that here that is not the casehere any program can have its input oroutput redirected because the input andoutput redirection is handled not by theindividual program but by the shell andso that way it applies to all programswithout any exception at all and in factthis goes a little further than youmight expect because not only are partsof the disk files as they are in othersystems but in addition the i/o devicesthe peripheral devices connected to thecomputer are also files in the filesystem for example the line printer andthe tape drive and even the thing thatdials telephone numbers are all devicesin the file system in the same programthat will copy information from one diskfile to another disk file will also copyinformation from a disk file to the lineprinter or from the magnetic tape driveto the printer the same program exactlya good operating system is easiest forprogrammer to use if the programminglanguage fits with the style of thesystem so along the way in the course ofthe unit systems development DennisRitchie created the C language C is avery nice high-level language with manyof the modern programming constructs init the thing that's very important aboutit is that it lets you avoid the detailsof the machine when you want to but whenyou need to and sometimes when you'rewriting an operating system you reallydo need to you can get at the details ofthe machine and control everything butyou're not forced to do that and that'simportant because that means you canwrite operating systems in this languageand still have something that can beportable to other machines the UNIXsystem has been moved to many manydifferent kinds of computers again thatmeans that people can ignore the detailsof what a machine is underneath and geton with their job now so at that level Cis by far the favorite language at thenext level the shell programminglanguage is very popular back on somemachines people find that the shellmeets all of their programming needsthey are writing lots of procedures tohelp them manage their work they don'teven have togo to a language at the level of see asit happens though because the system issuch a pleasant programming environmentprogrammers all over the world haveimported or added their own languages sofor instance you can find Fortran AlgolLisp basic fact almost any language youcan think of exists on some UNIX systemsomewhere what's important about theunique system is not so much what Ricciand Thompson put into it as what theywere able to leave out of it rather thanproduce a large number of primitiveseach one complex they were able tochoose a small number of simpleprimitives which could be fittednaturally together to accomplish complextasks this structure of the operatingsystem makes it natural and easy forpeople who create applications toproduce applications in that same stylefor example as the scale of integrationof silicon circuits gets ever larger wefind it necessary to have more and moresophisticated design aids to help peoplecreate large-scale integrated circuitsour existing design aids are advancedand effective but advances in VLSIcreate a need for even better toolsrather than produce those tools in theform of one humongous program designedto do everything the people here havebeen producing small packages eachdesigned to do some individual functionthat's helpful in the design ofintegrated circuits then theseindividual packages can be combinedusing shell procedures to design a partof a circuit or a circuit and the partsof a circuit can be combined to make awhole circuit Steve Johnson is one ofthe people who has been involved in thiseffort he is currently working on aprogram called Elgin which takes booleanequations as input and produces logiccircuit designs as output because toolbuilding is such a way of life on theUNIX system over the years we'vedeveloped tools that actually help usmake other tools these involve thingslike parser generators lexical analysergenerators and other programs that helpus organize and develop tools thesetools have been used in the developmentof L gen and many other applicationshere we see the boolean equations for asimple matter down here we have theequations for the carryout and the sumin the middle we have some descriptionsas to how we would like the cell to belaid out geometrically see we would likethe two inputs on the left side thecarry in on the bottom the carryout onthe top and the output on the right sidethe input equations first have to beprocessed so that they can be moreeasily represented in silicon thisprocess is very similar to recognizingcommon sub-expressions in the input of acompiler input language the booleanequations are read and processed by aprogram called yak which was originallydeveloped to help us build compilers buthas in fact been used in a large numberof application programs as well yak isbased on the theory of lalr 1 parsingrepresents it builds a small finitestate machine which is able to controlthe actions of the program which readsthe input detect errors accurately andstructure the input in such a way thatthe program can then go ahead andperform its operations on it after theseequations have been processedit's then necessary to worry about thegeometric layout of the circuit this isdone in the next 2 portions of el jemthe first program worries about theordering of these columns it uses atechnique called graph partitioning toattempt to iteratively come up with agood solution to what is in fact anextremely difficult problem in theoryafter the columns have been ordered thenthe tracks where the signals run arelaid out as well by another program andfinally in some sense we now have thecircuit designed and it's simply aquestion of realizing it with theparticular rulesfor our fabrication process and that isdone by a fourth program so once againwe have an example of taking a verycomplex problem dividing it into piecesrepresenting each piece with a separateprogram and then using the facilities ofthe UNIX system to glue the piecestogether into a coherent whole againcomputing is going to be more and moreinner woven with people's lives as theyears go by so computer technology isgoing to have to evolve to be easier forpeople to use the unit system is not theend of the road in this regard but Ithink it's a good step along the wayyouyouyou