HTML Can Sniffer

This is probably the most ridiculous way to reverse engineer something to date… a Chumby, an EasySync CAN Sniffer, PHP and JavaScript… but it works.

Instead of designing and building my own daughter-board for the Chumby, I bought an EasySync CAN USB converter which only has one CAN interface [and its not worth buying two].  This means I will have to use the Mazda’s MS-CAN bus only [That's where the fuel/trip computer information is].  Only one problem, I didn’t pay much attention to the MS-CAN bus before and the MS-CAN bus isn’t as interesting as the HS-CAN.

So new mission to find out what else is on the MS-CAN bus now.  Problem is, the nature of the information on the MS-CAN is different to the HS-CAN.  For example, something like Speed on the HS-CAN was pretty easy to decipher from just logging the data then plotting again.  Trying to figure out ‘door open’ for example on the MS-CAN is a lot hard this way.

I needed a tool to help me decipher the secrets of the MS-CAN, and I ended up writing (copy and pasting from old code) a ridiculous combination of PHP scripts for the chumby, a JavaScript/HTML front end.  All done and tested in 2 hours (lots of copy and pasting…) it works quite well.

How it works

• A background PHP script talks to the EasySync converter and reads the CAN messages that is being sent on the bus and makes this available to another data retrieval PHP script via sessions.
• The front end HTML/Javascript combination calls upon the data retrieval PHP, which feeds it XML on the messages for example the CAN Identifer and the timestamp.
• The XML is parsed and is rendered into a table for display, data is refreshed at periodic intervals.
• The Javascript compares the XML values with the already rendered values before updating to detect a change in value.  If there is a change, the background of the ‘byte’ field changes.
• A keypress monitor allows the user to ‘reset’ the changed background colours.
• Code attached below if you want to peek at the mess…

How I used it

Basically sat in the car with the sniffer running and trying various combinations to see which bit/byte changed.  e.g. Run sniffer, open a door, see message ’433′ byte ’0′ change colour, write down what changed, reset fields, open another door/do something else.

There were a few useful fields that I found/reconfirmed.  Nothing too interesting though and alot is disappointing.  For example there is information regarding headlight high beam being on, but not just normal headlights.  I suppose that’s because the high beam is displayed on the instrument panel whilst normal headlights doesn’t display/do anything.

Another disappointing thing is that the ‘ass/arse sensor’ (that’s what I call the passenger seat detection gizmo) is only a boolean signal…no pressure sensing in it at all.  Would have been useful if it was an analogue signal, could have been good for the mChumby display – “Warning, the girl on the passenger seat weighs 87kg, do not go on another date”.

I’ll be updating the new information to the Mazda CAN Bus page in the next couple of days… check back then to see what’s new.

Download cansniffer.zip (There’s still a bug with the timestamp I’m too lazy to fix… disclaimer this is hobby/crappy code )

I didn’t want to fiddle around with getting power to the Chumby, so the test is simply using my trusty old laptop and the EasySync adapter.  Only the speedometer, the tachometer and the accelerator pedal position is attached because again I’m too lazy to fiddle with my cable layout and as such its connected to the Mazda HS-Can bus only rather than the MS-Can where the goodie trip computer information is contained.

Almost every concoction of technology is used ranging from CAN, USB, PHP, HTTP, Shared Memory, XML, Flash, to an obscure programming language called HaXe.  Here is the result :-

The flash movie was set to 6 frames per second and the responsiveness of the data was superb.  I think I’ve stated before that the CAN speed data differs to the one displayed on the instrumentation cluster and this again really shows at higher speeds.

All I need to do now is to try it on a Chumby, there should be no portability issues as I’ve tested both flash and php on the Chumby before.

So does anyone have any comments and suggestions on the GUI and what else I could put on it?

Neat little package

The toys just keep coming in!  Another one of those wallet killing impulsive orders whilst I was sick in bed last week just arrived today, a USB-CAN bus converter module by EasySync that promises a great interface and should work in Linux!  This will give a little bump start to my Mazda CAN Bus / mChumby projects.

Note : EasySync UK’s online form doesn’t allow international shipping but a short E-mail or two managed that.  There is a US subsidiary with US and international shipping, but the international shipping rates on the US site is plain ridiculous [$68USD to ship a $88 device to Australia].

Note 2 : I hear the interface is mimicking the Lawicel CAN-USB interface…only hearsay…

Plugged it in – didn’t work.  Doh!  Took a peek at the manual and website, nothing on Linux other than “Linux drivers provided”… So much for the marketing material, guess I have to try make it work myself.  (Yes I got it working, read on for the quick run-down post aimed at helping others)

Sneak peek inside the box

Now with the photos out of the way, a little bit of detective work is ahead of us… why doesn’t this thing work… I paid so much money for it to replace the qprotos adapter!

A little bit of lsusb (or cat /proc/bus/usb/devices if you’ve got usbdevfs) sleuthing later…

Bus 003 Device 004: ID 0403:fac6 Future Technology Devices International, Ltd
Device Descriptor:
 bLength                18
 bDescriptorType         1
 bcdUSB               2.00
 bDeviceClass            0 (Defined at Interface level)
 bDeviceSubClass         0
 bDeviceProtocol         0
 bMaxPacketSize0         8
 idVendor           0x0403 Future Technology Devices International, Ltd
 idProduct          0xfac6
 bcdDevice            6.00
 iManufacturer           1 EasySync
 iProduct                2 USB2-F-7001 CAN-Plus Adapter
 iSerial                 3 ESNFWLPL
 bNumConfigurations      1
 Configuration Descriptor:
 bLength                 9
 bDescriptorType         2
 wTotalLength           32
 bNumInterfaces          1
 bConfigurationValue     1
 iConfiguration          0
 bmAttributes         0x80
 (Bus Powered)
 MaxPower              150mA
 Interface Descriptor:
 bLength                 9
 bDescriptorType         4
 bInterfaceNumber        0
 bAlternateSetting       0
 bNumEndpoints           2
 bInterfaceClass       255 Vendor Specific Class
 bInterfaceSubClass    255 Vendor Specific Subclass
 bInterfaceProtocol    255 Vendor Specific Protocol
 iInterface              2 USB2-F-7001 CAN-Plus Adapter
 Endpoint Descriptor:
 bLength                 7
 bDescriptorType         5
 bEndpointAddress     0x81  EP 1 IN
 bmAttributes            2
 Transfer Type            Bulk
 Synch Type               None
 Usage Type               Data
 wMaxPacketSize     0x0040  1x 64 bytes
 bInterval               0
 Endpoint Descriptor:
 bLength                 7
 bDescriptorType         5
 bEndpointAddress     0x02  EP 2 OUT
 bmAttributes            2
 Transfer Type            Bulk
 Synch Type               None
 Usage Type               Data
 wMaxPacketSize     0x0040  1x 64 bytes
 bInterval               0
Device Status:     0x0000
 (Bus Powered)

Yup definitely a FTDI device, even with the FTDI vendor ID (Did I mention I love FTDI?).  Looks like EasySync decided they wanted their own unique product ID, unfortunately that means the vanilla ftdi_sio drive won’t pick it up without a little bit of coercing.

The little bit of coercing goes like…

modprobe ftdi_sio vendor=0×0403 product=0xfac6

And bingo…

[ 6372.229046] usb 2-2: new full speed USB device using uhci_hcd and address 3
[ 6372.437247] usb 2-2: configuration #1 chosen from 1 choice
[ 6372.439834] ftdi_sio 2-2:1.0: FTDI USB Serial Device converter detected
[ 6372.439872] usb 2-2: Detected FT232RL
[ 6372.439934] usb 2-2: FTDI USB Serial Device converter now attached to ttyUSB0

Using putty to connect to ttyUSB0 at 115200, and typing “H” followed by “Enter” gives us

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2009.06.30 21:23:18 =~=~=~=~=~=~=~=~=~=~=~=
H
List of Supported Commands
O - Open CAN Channel
C - Close CAN Channel
t - Transmit Standard Frame
T - Transmit Extended Frame
F - Read Status Flag
V - USB2-F-7001 Version
N - Serial Number of USB2-F-7001
Z - TimeStamp Option On/Off
S - Set CAN Bit Rate
s - Set BRGCON Registers
m - Set Acceptance Mask
M - Set Acceptance Filter
B - Enter Bootload Mode
L - Set Listen Mode
E - Clear Buffers and Echo Char
R - Reset USB2-F-7001
H, ? or h - Help on USB2-F-7001 Commands

Only problem is that its Mac style line endings.  Even the append CR to LF option in putty is useless because the stream seems to be CR without LF…  Doesn’t hurt too much except displaying it is a bit annoying.

The EasySync USB-CAN adapter’s pinout is pretty much standard DB9 for CAN Bus and all I had to do was connect my existing J1962 (aka OBD-II aka DLC-2) cable on and onto the car I went.

A couple of short “S6″ (500kbps) “L” (Open connection for listen mode) commands later, I had CAN data being spat back at me

t04680601FF0000000000
t04680601FF0000000000
t04680601FF0000000000
t04680601FF0000000000
t04680601FF0000000000
t20F8FFFF27104000FF01
t2118FFFF000040000001
t4B082710271027102710
t19080000000000000000

All is good…except I just deleted my Chumby Development Virtual Machine a few hours ago…ooops…another day before I use this neat little adapter for mChumby.

I even ended up buying 2 Chumbies for the purpose of hacking one and developing the same speedo (The chumby is such a great alarm clock I couldn’t live without one…).

There has been quite a bit of interest in the comments for source code/information etc etc.  So now I’m officially starting the mChumby project!  This first post is basically a brainstorming session write-up or a mini requirements specification on myself.  Feel free to comment on any ideas/improvements you think I can make.

(This post seems too wordy without a photo but I’m just too lazy)

The concept

A Chumby will connect via CAN Bus to the Mazda to offer an alternative ‘GUI’ to the car.  Information such as fuel consumption that has been removed from the Australian Mazda’s will be displayed.

Since a chumby doesn’t have native CAN Bus connections, a custom daughter board with dual CAN controllers and transceivers will be built to replace the native Chumby power board.

The hardware

In addition to the Chumby mainboard, there will be a custom daughter board to be made to handle the car interface.  The Chumby has multiple SPI interfaces brought out through the ‘Chumbilical’, as well as on-board voltage regulators that will take up to 15V.  So a quick brainstorm on the daughter-board components leads me to :-

CAN Controller : MCP2515 – Has SPI interface, great for Chumby.
CAN Transceiver : TJA1041 or whatever it is I have in my box of electronics – Because I have them already
USB Connectors + Associated electronics…
Automotive Voltage Suppressor : Vishay had a few good ones

Luxuries like accelerometers are too much of a pain to hand solder

The software

I’m going to try use completely open source software in the development.  This will be discounting the Flash Lite 3 player that ships with the Chumby.  I debated much about developing a QT app or a flash widget, what I decided was undecided and I switched back and forth.  In the end, I’m sticking with Flash (for now).

There are two main pieces of software required, the GUI and the CAN Bus drivers.

The GUI as decided before will be Flash targetting the Flash Lite 3 on the Chumby.  For this I decided to use HaXe, which will happily compile to Flash 8 bytecode.  It is not as popular in open source flash developement as say ActionScript 2 with mtasc but I figured I’ll use the newer language.  Unfortunately there is a severe lack of HaXe examples on the net and when you find them, you’ll find that its based on Flash 9 API… GAH.  However a little bit of experimentation and googling around made me see how easy it is to actually use HaXe.  More on this later.

The driver’s for the MCP2515′s I haven’t really looked at yet but I hope that it’ll be a simple matter of adapting MCP2515 code from elsewhere and merging it with hopefully some SPI driver’s in the Chumby kernel source (did I mention how great the Chumby is? Source code is made available!).

There is a bit of other software that’ll be used to create this project.  I’m planning on learning how to use KiCAD to generate the PCB where as I’ve only previously used Eagle on my hobby projects.  While I’m sure Eagle will get the job done, I really want to give a try in learning how to use the open source KiCAD.  Let me know if you have any hints.

Geany will be my lightweight editor/IDE of choice, its so simple to use and amazing.  (I like Notepad++ in Windows but I haven’t booted Windows at home for a while…)

The HMI

I like easy to use and simple user interfaces.  This is especially true when I’m driving, I definitely do not want this thing distracting me.  My initial idea was to organise it such that all the ‘buttons’ are on the corners of the screen but on second thought even trying to touch that while driving may be a bit difficult.

Current idea is to use some simple swipe gestures to control the screen.  The idea is to have minimal interaction with my mChumby as possible while driving.  I intend to have 2 screens, one to display a large speedo and the main displays and a second screen to display a smaller speed with alot of additional potentially useless information (I haven’t figured out what yet).  I will also need brightness control to avoid getting a super bright screen at night.  So simply, swipe left-right will allow me to change screens and wipe up-down will allow me to adjust backlight brightness.

Below is just an embarassing sample of the HaXe code I was experimenting with…

Mchumby.hx

//MChumby GUI - HaXe code targetting Flash 8

class Mchumby{
  //Gesture Variables
  static var fGestureStartX:Float;
  static var fGestureStartY:Float;
  static var fGestureEndX:Float;
  static var fGestureEndY:Float;
  static var fGestureDeltaX:Float;
  static var fGestureDeltaY:Float;

  //MovieClips (Screens)
  static var mcMain : flash.MovieClip;

  static var mcD100 : flash.MovieClip;
  static var mcD101 : flash.MovieClip;
  //MovieClips (Buttons/Other Elements)

  //Text Fields
  static var tfText : flash.TextField;

  static var tfSpeedo : flash.TextField;
  static var tfAccelX : flash.TextField;
  static var tfAccelY : flash.TextField;
  static var tfAccelZ : flash.TextField;  

  //Text Format/Styles
  static var ts = new flash.TextFormat();
  static var tsCenter = new flash.TextFormat();

  public function new(){
    //Constructor - Most items are moved to main();
    fGestureStartX=0;
    fGestureStartY=0;
  }

  static function main(){
    //Setup main MovieClip
    mcMain = flash.Lib.current;

    //Set up some font styles
    ts.font = "Arial";
    ts.size = 24;
    ts.color = 0x00FFFF;

    tsCenter.font = "Arial";
    tsCenter.size = 24;
    tsCenter.color = 0xFF0000;
    tsCenter.align = 'center';

    //Set up screens the new way
    //var m100 = new M100(mcMain);
    //var m101 = new M101(mcMain);

    //Set up screens
    mcD100 = mcMain.createEmptyMovieClip("D100", mcMain.getNextHighestDepth());
    mcD101 = mcMain.createEmptyMovieClip("D101", mcMain.getNextHighestDepth());

    //Set up text fields
    tfText = mcMain.createTextField("tfText", mcMain.getNextHighestDepth(), 20, 20, 280, 100);
    tfText.setNewTextFormat(tsCenter);

    //Set up text fields (D100)
    tfSpeedo = mcD100.createTextField("speedotext", mcD100.getNextHighestDepth(), 20,50,280,100);
    tfSpeedo.setNewTextFormat(ts);
    tfAccelX = mcD100.createTextField("accelx", mcD100.getNextHighestDepth(), 20,80,280,100);
    tfAccelX.setNewTextFormat(ts);
    tfAccelY = mcD100.createTextField("accely", mcD100.getNextHighestDepth(), 20,110,280,100);
    tfAccelY.setNewTextFormat(ts);
    tfAccelZ = mcD100.createTextField("accelz", mcD100.getNextHighestDepth(), 20,140,280,100);
    tfAccelZ.setNewTextFormat(ts);

    //Set up text fields (D101)

    //Initialise text
    tfText.text = 'Hello World!';

    tfSpeedo.text = 'Accelerometer Values...';

    //Set up event handlers
    mcMain.onMouseDown = startGesture;
    mcMain.onMouseUp = endGesture;
    mcMain.onEnterFrame = update;
  }

  static function update(){
    tfAccelX.text = Std.string(ChumbyNative.getAccelerometer(5));
    tfAccelY.text = Std.string(ChumbyNative.getAccelerometer(6));
    tfAccelZ.text = Std.string(ChumbyNative.getAccelerometer(7));
  }
  static function startGesture () {
    fGestureStartX = flash.Lib.current._xmouse;
    fGestureStartY = flash.Lib.current._ymouse;
  }

  static function endGesture () {
    fGestureEndX = flash.Lib.current._xmouse;
    fGestureEndY = flash.Lib.current._ymouse;
    fGestureDeltaX = Math.round(fGestureEndX - fGestureStartX);
    fGestureDeltaY = Math.round(fGestureEndY - fGestureStartY);

		//Rather than else-if, the following if's are deliberately done to allow compound gestures
		//e.g. Up & Left, Down & Right etc.
    if (!Math.isNaN(fGestureDeltaY) && (fGestureDeltaY > 100)) {
      gestureDown();
    }

    if (!Math.isNaN(fGestureDeltaY) && (fGestureDeltaY < -100)) {
      gestureUp();
    } 

    if (!Math.isNaN(fGestureDeltaX) && (fGestureDeltaX < -200)) {
      gestureLeft();
    }

    if (!Math.isNaN(fGestureDeltaX) && (fGestureDeltaX > 200)) {
      gestureRight();
    }
  }
  static function gestureRight () {
    tfText.text = "To the right!";
    mcD100._alpha = 0;
    //tfText.setTextFormat(ts);
  }
  static function gestureLeft () {
    tfText.text = "To the left!";
    mcD100._alpha = 100;
  }
  static function gestureUp () {
    tfText.text = "Bright!";
    ChumbyNative.setLCDMute(0);
  }
  static function gestureDown () {
    tfText.text = "Dim!";
    ChumbyNative.setLCDMute(1);
  }
}

ChumbyNative.hx

class ChumbyNative{
  public static function getLCDMute():Int{
    return flash.Lib._global.ASnative(5,19);
  }
  public static function setLCDMute(x:Int):Void{
    flash.Lib._global.ASnative(5,20)(x);
  }
  public static function getAccelerometer(x:Int):Float{
    return flash.Lib._global.ASnative(5,60)(x);
  }
}

Mchumby.hxml

-swf mchumby.swf
-main Mchumby
-swf-header 320:240:12:000033
-swf-version 8

There are other rival bus technologies include LIN (Local Interconnect Network) which is supposed to be a lower cost alternative to CAN, but I’m not interested in that.  Why?  My car uses the CAN bus, not LIN, not Ethernet.  So lets go about figuring what can be gleened from the communications whizzing around my car!

Aside from CAN bus, my car has a pig too (Long story). This photo is from 2 years ago and the pig no longer has the weird thing hanging from its belly or the tag on its ass...

Background / Reason Why

In June 2006 I bought a ‘new’ car, an Australian Mazda 3 Facelift Model.  The car was a great drive, Mazda excelled in the build and features but one thing nagged at me – the speedometer.  It was stylish I had to admit but the scaling was ‘off’ to me.  0 speed was vertically down and the scaling was… better not rant on but the thought was ‘maybe I can get another speedometer on’.  A history of my ‘better speedometer’ fantasies follow

My old (and free!) Sony Ericson showing my speed on a MagLev

Testing the feasibility of using the same crappy phone as a HUD on another car of mine on a rainy night...yup that's the image on my windscreen!

A little bit of research later I found out the Mazda had two CAN buses: a ‘HS-CAN’ (High Speed) network on which most of the essential systems (ECU, ABS) were connected, and a slower ‘MS-CAN’ (Medium Speed?) network on which accessories such as the audio system were connected.  These two networks are joined together by the instrument panel which also acts as the gateway to allow some messages to pass between the two networks.  Both buses are conveniently available via a ‘standard’ J1962 plug just below the steering wheel (Do a search on OBD) so I didn’t have to hack anything open.

[Note : Most of the information below is from back in 2006, I only recently picked this project back up]

[Note 2 : Please don't any of what I say below if you're not familiar with electronics/cars or if you value your life... I hold no responsibility for anything that happens... I need to learn how to write a better disclaimer]

So what is on the network exactly?

Officially?  ‘I dunno’.  Mazda naturally doesn’t publish this information nor can it be found on the net easily, I hope Mazda doesn’t take offense at me writing this post actually… I loved the car and I’m doing this because its such a great car [/grovel].

Now if that information isn’t forthcoming from the great ol’ google, I just had to try figure it out for myself!

How it was done

First step – Find the pin-outs of the J1962 connector, since its standard connector albeit with slightly different pin-outs per car type the answer is easily found on google.  Go search it up yourself

Second step – Oscilloscope.  A lovely portable fluke connected to the two CAN bus lines told me the baud rates of each of the two buses.  500kbps and 125kbps respectively.

Third step – Start sniffing, or at least buy the stuff to do so.  I had wanted to build my own CAN interface board but laziness got the better of me and I bought the cheapest one online I could find from QProtos.  In hindsight it wasn’t the best choice but it wasn’t that bad either.  Coupled with a J1962 cable from OBD2Cables.com (only remember them because i’m still receiving email offers hehe) I had everything I need to start.

Fourth step – Really start sniffing.  A precaution I took was to set the CAN-USB adaptor to ‘listen’ mode only, I didn’t want to risk the brakes suddenly not working.

A bunch of static tests to start things off nice and slow.  The methodology was simple, start the CAN bus logging then carry out actions on the car at certain times – turn the blinker, step on the accelerator etc.  All was left to be done was to review the logs to see which bits or bytes changed at those matching times.

Fifth step – The fun bit!  Dynamic testing!  This time with the same setup as above but with my GPS phone logging simultaneously too.  Oh what fun you can achieve here.

Speed from GPS vs Speed from CAN bus

Anyone familiar with the Australian Design Rules and the laws?  The displayed vehicle speed must not be reported lower than actual, so what most manufacturers do is artificially bump that number up.  Hence you see the almost always consistently higher CAN speed compared to the GPS speed… Hey I haven’t even said how many ‘speed’ signals are transmitted across the CAN bus…the ‘actual’ (unmodified) one is there too.

The state of affairs

As of the end of 2006 (Yes two years ago) I had mapped out … (in no particular order…well actually in order of CAN bus ID’s)

On the HS-CAN

• Brake Active
• RPM
• Vehicle Speed
• Accelerator Pedal position
• Brake Active (Again)
• True Vehicle Speed
• Gear Position – Park
• Gear Position -Reverse
• Gear Position -Drive
• 1st, 2nd… etc Gears
• Gear Shift
• Manual/Tiptronic Selection [Yes I drive an Auto]
• Gears again…
• Accelerator Pedal position again…in a different scaling…Mazda is strange
• Left Front Wheel Speed
• Right Front Wheel Speed
• Left Rear Wheel Speed
• Right Rear Wheel Speed
• Steering Wheel Angle (I’m going to have fun with this one!)
• Odometer

On the MS-CAN

• RPM (Retransmitted)
• Vehicle Speed (Retransmitted)
• Head Unit LCD Display (Oh this will be fun too)
• TRIP COMPUTER! Average Speed, Fuel Consumption, Average Fuel Consumption, Distance Remaining

Oh I so forgot to rant.  Trip Computer, very nice little pieces of information for driving a car.  Official Mazda Australia line?  Oh our Mazda 3′s don’t have this.  All overseas models have it.  All information is already on the car.  The car even has a head unit with all the trip computer LCD displays present (you can see it if you tilt your head).  Infact there’s a little trick you can do with every Mazda 3 model just prior to mine that you can hold down certain keys on the head unit before starting the car that activates it… but nooooooooooooo its disabled on mine for what reason I don’t know… erm… rant over.  Guess I have to do a better speedometer and trip meter too.

If anyone was expecting to see the results, you have to wait for Part 2 (or 3)…does anyone actually want to see?   Or got any questions? mmmm