Monthly Archives: March 2010

The Royal order of the Mechanical Masonic Sheep

The last few weeks have been a rather busy blur. This started with the Nova Albion Steampunk exposition. I just returned from mars. More specifically the Contact conference presented by Cultures of the Imagination.

Contact, is an interdisciplinary gathering of Anthropologists, Writers artist, Filmmakers and Engineers. The last few years this has been held on or near the NASA Ames Research Park in Mountain View, Silicon Valley. It may be easy to describe this as the oldest and best known SETI conference. This just may be an excuse for a lot of bright talented people to get together and party. No one has yet to see a little green man (or woman) so, there seems to be some job security for those who like to listen for them.

This year the conference was more focused on the anthropology side. Possibly due to the economy there seemed to be an absence of Science Fiction writers. So get out there and start buying more books. As a robotisist, I tend to hang out with the NASA engineers who get to drive sporty little rovers on the other planet. This is a fun group, and I get to compare how much dust collected on the toy rover I got when Spirit and Opportunity landed, to how much is on the real thing,

MER exploring my messy desk

From Pixar, we had the showstopper of a presentation, The latest in photo-realism. With a series of stills, it was difficult to tell what was real and what was fake. The highlight was a 10 second clip from an introduction to what is possible. This could have been anyone recruited from a temp agency to give a short product pitch. That the product is the pitch makes it only all the more a little closer to the cliff called the uncanny valley. Masahiro Mori[1] proposed this theory as an explanation for the likes of Doctor Coppelia.

Sometimes it takes a strong pair of Doctor Coppelias’s rose tinted glasses to see into this. Even I have found some of my puppets and dolls a bit disturbing. I am not exactly sure what they are up to here. This does bring in some interesting connections to the need for 3D eyeglasses, to complete the illusion. I suggest that one should not get to attached to my friend Olympia. On the other hand frequent older readers to the main, site, have been able to use such rose and cyan tinted glasses to see into the precursor of Olympia, the doll sometimes known as Marianne Droz.

The french really do love Ben Franklin

The most surprising Contact, I made at the conference, was with a Marine Biologist who has done some amazing studies of human dolphin communications. Pop culture films from the 1970s aside, this may be our first contact with a non human equally intelligent species. Something well known to the likes of Homer and the great engineer and puppet maker from Alexander, my own hero, Hero. Here is a snapshot of his take on clockwork dolphins, who take a liking to Ajax.

It is all Greek to me

So leaving this fuzzy world of mars I next attended the AMICA meeting.

Now I love mechanical music, Those rocking rags and blaring marches from the 1910s and 1920s. This is much like fine scotch whiskey. It is an acquired taste. I also like Doctor Who, Puppet show and Mechanical dolls. My favorite episodes of this 40 something year old TV show involve these somewhat uncanny things. The more recent ones written by Stephan Moffatt. Recently he wrote about why actors from over the last 47 or so years should not return:

“… sometimes that can make your new and most cherished viewers feel excluded from the party. And Doctor Who has to absolutely be a brand new show…

…The moment it becomes something your dad wants with old codgers wandering about, it will die. It is not important to bring anything from the past. You could bring nothing back from the past ever again in Doctor Who and it would be fine.”

I kept thinking of this quote all weekend. “Something your dad wants.” Is this why some say Mechanical Music is doomed? Because it becomes about the music. Ironically, the last gasp of the first generation piano roll production ended with the Beatles in the late 1960s. These guys all passed on. The middle age kids of the time are now the old codgers of today. Ironically most of this generation do not like the blaring of John Phillips Sousa. But it is what they associate with going to the beach amusement park and hearing the rusty old squeezebox blare it out.

Now AMICA sometimes seems like one of these collections. Covered in dust an echo of a forgotten generation. The problem is that the older generation wants to preserve “Their way.” The conservative idea that the world does not change. That children are always the same and must be taught simple things before they become bored. Of course in this day and age, the Disney brothers would be locked up for frightening children and giving them nightmares.

Much of the broken fairgrounds junk was thrown in the trash. What remains is rare and sells for large sums. There are about a dozen mega rich collectors who hoard this stuff in their Mc Mansions, along with auto collections and the like. I say more power to them, the have preserved the world of their youth. Of course outside of places like San Francisco’s Museum Mequanic, children are not allowed to view these toys.

There are cultures where these older mechanical toys are taught to the children. According to one person I spoke with over the weekend, that many Americans are now working for these children. That there race is now somewhat smarter than our race. I do not agree with this assessment. It takes a lot of brainpower and intelligence to be ignorant and dumb and work the social welfare systems inside these tribal family groups.

Today those of us who are makers, some who identify in the steampunk moment, are creating new concepts from these trash heaps. I really wish there was a way, where Makers, Steampunks, Re-enactors and mechanical music enthusiast can come to a common understanding.

Now I have been spending the last month or three reading the history of the Panama canal before bed. This mess was the biggest engineering disaster the world has ever seen. The last of pharo’s gangs of labors toiling for the masters. Actually the canal is quite small, Smaller than San Francisco bay. Modern freeway projects move more dirt. and all that. So what does this have to do with mechanical music?

Well once the mosquitoes were abated by socialism, The white overloads who worked for Caesar Imperious , were able to create secret society clubs. These were all the thing back in the 1880s and such.

So I have decide to create the Secret Society of the Royal order of the Mechanical Masonic Sheep. Membership in this of course is secret. So only those who know that they belong can join. However this is an Open Source masonic order. So therefore any work done by the Mechanical Masonic Sheep, must be open for others to access. This by definition means that everyone is a member of my club. No one is or can ever be excluded.

The first rule is that there is not extortion to be had from other members. To counter this members must be helpful and donate information, to those who do not have it. No member can fine or tax any other member without just compensation. On the other hand members are free to contribute ideas time and material wealth to others. As everyone is automatically a member, this rule is completely voluntary. I am sure most will ignore it.

The tone of this order is modeled as a mash up of Mozart’s Magic Flute with Gilbert and Sullivan’s Iolanthe. So this is also a fairy masonic order. To keep things in line with the crazy weekend, Benjamin Franklin will be the patron of the order. Homer’s Odyssey the framework of the mythos. The god of the order are Isis and Osiris. And my own Palas Athena Governor. The bay Laurel is the official tree, and the rose the flower. Descartes is the philosopher of choice.

Of course this is all nonsense, most of the world is. I set out write something political in this blog and feed into the frenzy of the changing times we are witnessing. Instead I get more abstractinos from practical real world experence.

For me 100 years ago will always be the 1850s and 1860s. Because that was what I learned as a child half a century ago what one hundred years meant. 200 years was the 1770s which were exiting times in the world of mechanical automata, painting and just pure design aesthetic. There is a reason 2001 is my favorite film. I want my clockwork Heaven. The crystal spheres, silver gradients and the brass gears.

Now we find that 100 years is the age of the Panama Canal, The Titanic and ragtime music. A world many want to return to with a divided class labor force. You are as you are born. This is your cast. The height of the Raj of India. One god indivisible with justice for all. Not that I can improve on any of this. I decided a long time ago that I did not like the rotted well water of politics. This does not, invalidate my theory that with politics, is much like a well. That there must be a fair amount of brackish water on the bottom for the sweet water to flow on the top.

Drain the well of taxes and socialism too far and one is left with the bitter taste of bad water. Now I am rather selfish when it comes to politics and opinion, these are mine and You can not have them. Still I tend to be a liberal hawk. Probably from my upbringing inside the Silicon Valley industrial complex. Johnson’s Great society. How Taxes are supposed to work. What does it really matter after 100, a thousand or so of years, If people are lazy or not. Everyone contributes. Some philosopher said “to each according to his need.”

This is how the panama canal was built. How we went to the moon a half century ago. This is also how renaissance fairs, woodstock clones, burning man, science fiction conventions, Rock concerts, and a whole lot of successful things have and will be done. This is what happens when society collapses as the Media is us is soon. After reading Ken Follit’s Pillars of the earth, this may also have been how Cathedrals were built.

What is interesting, is that while there will still continue to be backlash at change, Some inspired some done by stupid methods. There are already forces at work, like the Royal order of the Mechanical Masonic Sheep. That counteract these. Much today would be like people in the 1920s refusing to use electric light, because god intended humans to see the light through candles and rush lights. Perhaps this is why some cultures would like to return to oil lamps. Rub them and the genie appears.

But the genie is already here. Some call this science. The problem I have as an abstract thinker, is that I have seen no evidence of the parental god. The one who punishes when bad and rewards the good. This is not to say that I do not have faith. Here again I am selfish, as my faith is not yours and you can not have it. Can god be simplified in the equation 2+2=4? Descartes though so, when he defined the sign for addition as the holy cross. The equals, shows the view of Solomon, with man below and god above.

So no matter what happens, god will always be with us in science. But should this be kept in ignorance? I am a Pan-deistic,Poly-theist. Or all the gods all the time. No one is right and no one is wrong. The real point from a conference like Contact is just that. What we have is each other, and after an almost infinite amount of time, rocks can sing to the sky, just as birds and fish can live together inside the imagination of the human or abstractly similar mind.

A short history of the Midi filer project from floppy disk to 8GB SD

This is why I never blog. I get so involved with projects, there is never anytime for anything anywhere. Recently I set about to salvage a project from 2003. If I had been smart I would have blogged the whole thing. Who was to know the future.

Many years ago, or is it really decades ago, I started a project to read MIDI files from floppy disks. As luck would have it, someone actually wanted to pay me to do this. So off to Kentucky I went. The project was a disaster. Many people claimed ownership and the funding failed. After three to five years of development, the shipping date was always 6 months away. In the end they shot the horse.

In the middle of these near fist fights I was able to get the prototype unit working, which read Floppy disks in the popular MIDI formats, Sandard type 0 as well as the Yamaha and Pianodisc variations. This technology I was given as the partner I direclty dealt with wanted to have nothing more to do with it.

One of the other partners suggested I continue with the design. Several clients indicated an interest in another source of midi filers. I was given the specs for the Yamaha MDF-3 as a guideline. I had envisioned a system that would read the compact flash cards. This was actually suggested by my dad.

When I first took the project back to California, I set about to extend it to read Compact flash cards. The main controller was a DS80C320 chip. This was attached to a PSD and 128KB of external memory. Much of the issue the prior developers had, was getting enough data pins out of the system. To fix this a separate Atmel processor was used to scan a button matrix.

My first effort was to improve the button matrix. This I moved to a slightly larger processor (Still Atmel.) This was actually made. I was not given any budget for new main boards. These were large with a lot of dead space on them. I managed to make these things work. The main code was in C, with the button scan in ASM. While the 80C320 has two serial ports, these have to use the same baud rate, which was determined to be the MIDI baud rate.

The little mega 4433 ran circles around the 20mhz 80C320. It looked like a full midi parser would fit. I could greatly reduce the cost by 2/3 if I went only with the atmel chip doing everything. Atmel even made one with the same pin layout as the 80C320. The choice was further defined, when the first partner demanded all his hardware back. The second partner was still supportive, So I returned the hardware to the first and re-designed the whole thing using only Atmel. Only the 128KB memory remained. The second partner continued the funding.

I set a goal to have a dozen units ready for the next MBSI meeting. It was a mad dash. The boards were ready in time, but the program still had issues. I put some photographs from this into the Gallery here. I was surprised to see the corner of the old Overture monster behind my then state of art unit. I was over confident and did not include a debug port. I also made a mistake on the LCD controls, which caused a glitch every time the display was written. I also learned a lot about PC board layout. Fortunatly there are some good online resources such as the EDA tech forum and the conferences and daily updates I get from the UP media group. Check out the Update Newsletter I need to give them a plug every now and then.

Three months after the MBSI meeting a few units were shipped. These worked well. Over time I extended the system to read SMF1 files as well as SMF0 files. By the time I had this working floppy disks were dead. I was also deep into rebuilding the Caliola. The project was such a disaster I wound up out of cash and back living in my parents basement. No more trips to Swizerland.

I still wanted a compact flash reader. I figured to use the file parsing code written for floppy disks. Reading the CF flash card was easy. I quickly reduced the design to fit on a small proto board. The dozen or so unsold floppy units still a bit of a bother. I am a backroom engineer. The LCD display was too big for the case, so it sat on top. At this point, I did not want to spend any more on the floppy box. It worked, but did not look pretty. I have no idea what ever happened to the 5 units I sold. The users never complained, and indicated that the boxes were functional and worked.

I was working with Spencer Chase on improvments to the e-valves. Spencer assisted with the development and the 3 units, of the CF card versions sold quickly. One is in regular use, and has been for a number of years. I took back the Spencer’s to upgrade and it has taken some years to get back to it. I have no idea what happened to the second unit.

In the image gallery, one of these “Toaster” units is shown. I found some old photographs, one which shows the large floppy board next to the “mini” unit for reading Compact flash cards. I could not afford a FAT32 card and the small FAT12 cards worked funny. The solution was easy, to add one more address line and read the disk sector as a memory map. Easy to debug. set the registers and the AVR studio debugger would show the sector. Bit dicey though as the bytes have to be read in order

Other projects intervened, and I pretty much decided I had done with MIDI filers. As with all dead end projects, I figured to revive them somehow and sell off what assets I could in time. The little CF readers are cute, and can do a lot. I was more than happy to revive and make a new batch for Larry Doe and the Keystone. The catch was that MIDI in was needed.

So the code was completely re-written. After some extensive testing, a unit was shipped. This had a problem with the system resetting during the perforation. While updating the companion MIDI “primary” controller board, I discovered and issue where input collisions happen in the interrupt even though the flag is technically clear.

Now for the pea soup course. In early 2009 I learned about the reprap project, which is a 3D printer. As I was installing a Theater organ and building a 20 note busker, I did not have the time to look into 3D printing. A desirable device, when one wants pretty cases for electronics. Then an old friend wanted a “primary” board. Sell the rest on eBay, he suggested. Even that was two years ago.

Back to the Reprap. The hardware on this is almost identical to my Midi filer, and the MIDI “Primary” boards. Same processor. There is also someone in Germany selling a system using SD/MMC cards. The boards are full of SD “How to” postings.

Is SD really that simple? Compact flash was trivial, but it needs a lot of I/O pins and a large layout on the board. SD on the other hand promises to only use the same pins as the system program interface. On the Floppy midi filer, I used some of these pins for the key matrix. Could one be modified to use SD instead of a floppy?

I built a reprap motherboard with SD. Even started to convert my 8K fat reader from MIDI to G-code. Yes, Sophie, I am actually getting to things that happened this month! So I went back to the old unsold floppy filer boards. That lack of a debug port really hurts. If I had 17 seconds to do over, I would have put that footprint down. The CF boards have a debug port, But the Ice200 broke, and the mega162 put the JTAG on the same pins I use for the CF DMA. Another dead project revived. Using stuff about the house I made a USB adapter. Ice200 comes back to life.

Many time I have attempted to fix the issue with the LCD on the filer board not being memory mapped. Backing up a bit, I went over the logic again. On the CF version I used a 74LS138 as I should have in the first place. With only two selects on the filer and one a select high, I went all cutesy and did it with logic gates. The problem was I was always one inverter short.

I fixed the filer, by adding in gate to OR and invert the readwrite line. This was all and well, apart from that the memory mapped all over the lower 32K with duplicate windows everywhere. I never liked this fix. it did not work with the mega162, which is needed if one wants to write MIDI as well as read MIDI.

A few days ago, I figured it out. By combining the read write invert with the address invert, I am able to fit the needed logic into the two existing chips on the filer. I also decided to move the selection back down to the lower memory. In debug mode the m162 only has a tiny window for memory mapping. Even with the current fix, the DMA takes 256 bytes per address. I really do not like switching on the latched memory buss pins. The 128K memory is sensitive as it is.

Now it is possible to write fat code that fits into 8K and in C too. At least for FAT 16 cards. On the other hand I have the 128K boards and I really need to sell them off to make the world better and people happy. So I set out to rework the filer boards. The first was to make the CF filer board the same as the floppy filer. This to test the LCD logic.

A new issue arose, The button matrix, which dates back to the old 4433 processor, used the system program pins. The LED indicators were always an option. So these could be done away with. This left two lines to be used by the matrix. Two lines also needed to detect Card insertion and write protect. As the chip is full, no writing is possible, so that can be ignored. Detecting the card is done at power on, so this can be a poll on the card ready bit. — Then an epiphany. Play is often combined with pause, so that frees a location in the key matrix. This is perfect for placing the write switch such an input can be shared with the record logic. It is also possible to add the card detect to the key matrix. A problem, that has bothered for some time suddenly disappears like an illusionist trick. The problem never existed in the first place.

So the lashup was complete. The logic worked on the mega 162 board. Fixing the Ice200 and connecting the SD card to the system program pins. The magician stands ready. Nothing up the sleeves he says. And the directory loads into the external Sram. Changes are made, The 8GB card comes ready. The directory loads. One single cluster, 32K fills one bank of the 128K external memory.

What a sight. The entire contents of the first developments Hard drive was only two gigabytes. Here it is loadable into what was originally a system for reading tiny little floppy disks. Every midi file I have takes a small fraction of this card. It is a lot of button pushes to get to any one of them, at least I implemented hierarchical directories.

Of course it is only a magicians trick, The code sees the 32K sector wrap to zero and the system crashes. Sill the 8GB are readable. As all ways there is more to do. Sectoring logic to be written so that one can step through hundreds, if not thousands of files. The SD code is also a bit more complex than CF, I had to switch off FAT12 and Piano disk support to add the init code in. The floppy and CF access was DMA, so a few bytes to set a register and then read the DMA window. SD requires almost 64 bytes of tables to be saved which detail the commands Used for card type detection. While a lot of these are zeros, it takes two bytes to load them into a register, where as if stored as a table, the read loop takes only a handful of bytes.

There is still much more to do, But the main issues are resolved, The old boards awakened from sleep. Not Zombies or Gouhls, these boards were never finished, the unmade, or unborne fairies of the aiether. Soon to come forth, with new dresses, and fly on the desires and dreams of all who want a dependable and reliable way of saving more MIDI music than can be had at any cost.

Rebuilding the UM1

Some years ago I was given a UM1 that had the logic supply connected to enough voltage to crack the power regulator. Most likely every chip was blown apart.

I had used a similar processor for a video conversion project, which has long sat documented in the video section of my website, an early form of blogging from the late 1990s. I still had a tube of these processors, although the ones I had were 52 pins, where the processor on the um1 had 44 pins.

Since I frequent the surplus shops of the silicon valley, I located most of the chips used in this board. Given supply issues, I have for other clients developed a similar board using a more modern embedded micro. More on this in a later blog. The short version is that I had a UM0 and wanted to keep the same setup codes and configuration on my Caliola.

By connecting a MIDI sniffer to the wire, I was able to detail how the setup system exclusive messages worked. Technically these are supposed to be documented in the manual. The manufacture is not registered so the device setup for these exclusives is the research mark. Which means that this data is open and unprotected, so I could use it.

I noticed that the Octet setup program had a mechanism for uploading new firmware updates. These updates were available on the web. When opened they were found to be in the common format used by the MC68HC11. There was even included a small program called a talker.

Locating the needed chips, the question remained, would it be possible to reprogram the actual UM1 unit from the web updates?

The first step of the process was to remove all the old chips. This was done with a heat gun. The board is slowly heated to 210degrees C. My thermocouple slipped and some of the sockets got a little cooked, the result was a clean board. The front panel was the worse for wear as the switches melted completely.

After the chips were removed, the traces were cleaned up and documented with the Eagle layout program. Oh for the days of Sam’s Photofacts. I wonder if the library still has the shelves of 1950s and 1960s era television schematics. Actually I was able to get schematics for my late 1990s Panasonic, telly. So major manufactures do still provide repair personnel this needed information.

The designer of the UM1 had been dead for some decades now. The successor does provide manuals and firmware files as noted. I took a peek inside these files and located where the device tables go. In order to bring a new system up from scratch, a few additional items such as the serial number are needed. This was easy to get, as it was pasted onto the blown up chip. The firmware for some reason does not contain the device string, which for some reason is always UM1 even on a UM0.

I was a bit concerned that I would have to solder in the new processor. These are surprisingly still available. This package called a PLCC is no longer made. When the EU adopted RoHS this format was discontinued. In the surplus store I was able to find a socket that would solder onto the board, so anyone in the future who needs to fix this can swap the chip out.

Once the new chips were in place it was time to reprogram the unit. I could have connected in through the MIDI interface. The 68HC11 contains a serial boot mode. There is a photograph of this in the attached image gallery. These signals are easy to access on the 74HC02 chip labeled U9. For future reference the pins are Rx on pin 8 and Tx on pin 6. Laurent must have used jbug as the wiring of these pins is straight out of the college textbooks.

The 68HC11D0 contains no internal flash memory. It is easy from looking at the schematic, to see where the memory starts. 0x2000. This is where the jbug11 talker is loaded. The firmware loads in at 0xE000. This is where the SYSEX tables are stored.

The first step is to upload the firmware from the website or setup disks(If available) This can be uploaded by simply renaming the file extension from .frm to .s19. Since the firmware does not overwirte the device information, this will need to be copied from a working UM1 The format is exactly the same as what the UM0/UM1 setup software sends, so this can simply be done by dumping the read firmware sysex

Once this data is written to the first sector of the 0xE000 address, there should be enough code to get the whole unit working. In my case I replaced the MIDI opo-isolator and the U9 74HC02 chip. I was able to connect using the Octet setup program. Just to be safe I re-ran the firmware upgrade.

It should be obvious that a working UMx is needed to get things stared, this blog is about repairing units. There are much better and cheaper ways to make similar devices, which is why my own boards are only functionally equivalent, with no common hardware.

Naturally the diagnostics did not pass as I had not seated in the driver chips. These are often out of stock and I had to get some from other units for testing. Once installed, The diagnostics pass muster and the unit is once more operational apart from a missing chip for the PWM and the melted front panel. The UM1 works fine without the front panel, but there are some unsightly holes where it goes.

Repairing the front panel did not have quite the same success. I socketed these chips, which means the display plate no longer fits. The surplus hex switches are also much larger and do not have the same pinout, so give the wrong midi channel. There is also quite a bit of flicker on the LeDs. Most of these units are set once anyway, so the front panel is a nice extra, but not needed.

It is interesting to compare my “clean room” copy to the original. There is no hardware in common. I do parse some of the same setup code which is what makes my unit a clone.

The repair on this is also quite expensive, The obsolete processor sells for 10s of dollars, when in stock. [Currently at this I have some similar E2 processors for sale on eBay]. Atmel processors sell for a few dollars each, These contain many times the storage space and do not need the external logic. The replacement parts on the UM1 were around 35 dollars not counting the driver chips. It probably would be cheaper to buy a new unit, if such were available.

Over the next few months, I expect to sell some of my compatible prototypes online. I much prefer consulting and building devices to order. I also am willing to take on repair of the octet systems as well as any Broadmore power rolls that may still exist.