See Saw Photo

A quick note to let people know that if you need the best photographers in Brisbane Australia, go straight to See Saw Photo - run by my brother and his wife. Their portfolio is amazing, and if you appreciate good photos, it's worth a look just for that.

A new USB tutorial coming soon!

USB Part 2 - The Joy Mouse

In this example, I’m going to show the use of the library with the TechToys USB experimenter’s board. This is a great board to try these sorts of experiments with – it has an SD card slot to play with mass storage class devices (along with understanding the SD card protocol) and a 5 way joystick, supporting up, down, left, right and select. This allows us to simulate a mouse with our joystick. Since the mouse is part of the USB specification that Windows supports without (third party) drivers, this is a nice way to get our feet wet with USB without having to do any hard work on the Windows side. You could also prototype this circuit using a 18f4550 pic on a breadboard with a few switches if you preferred.

Let’s look firstly at the way we handle delivering the descriptors to the host. Pull up the usb_joy_mouse demo and have a look in the usb_config_mouse.c file. Here’s where all the mouse specific stuff sits to hide it away from the main program.

void usb_get_descriptor_callback(uns8 descriptor_type,
                                 uns8 descriptor_num,
                                 uns8 **rtn_descriptor_ptr,
                                 uns16 *rtn_descriptor_size) {

You have to supply a usb_get_descriptor_callback function in your program. The PicPack library will call it when it has received a request for a particular descriptor. Your job is to return a pointer to where to find the descriptor, and how big the descriptor is.

The standard descriptors are defined in pic_usb.h. Here’s the device descriptor:

typedef struct _device_descriptor {
    uns8    length,
            descriptor_type;
    uns16    usb_version;    // BCD
    uns8    device_class,
            device_subclass,
            device_protocol;
    uns8    max_packet_size_ep0;
    uns16    vendor_id,
            product_id,
            device_release;    // BCD
    uns8    manufacturer_string_id,
            product_string_id,
            serial_string_id,
            num_configurations;
} device_descriptor;

Now, the descriptors really are just a chunk of bytes, so you don’t need to use proper C structs like this to hold them. You could happily use a string of bytes and return a pointer to that along with the length. The reason we use C structs here is that you are much less likely to make a mistake getting things working. Once you have things working, feel free to replace the C structs with data structures that take less space or are based in ROM (with appropriate changes to the library). As always, my motto is get things working, then get them working smaller/faster. Did you know you can hang Windows by plugging in a device with a dodgy descriptor in it? I didn’t, until I started this USB work. Believe me, it’s a frustrating exercise! Who would have thought that Windows would be so easily duped? So, in these examples, we always use the structs to ensure we have everything right in the descriptors

Now, back in usb_config_mouse.c, we define our device descriptor like this:

device_descriptor my_device_descriptor = {
    sizeof(my_device_descriptor),     // 18 bytes long
    dt_DEVICE,     // DEVICE 01h
    0x0110,    // usb version 1.10
    0,        // class
    0,        // subclass
    0,        // protocol
    8,        // max packet size for end point 0
    0x04d8,    // Microchip's vendor
    0x000C,    // Microchip's product
    0x0200, // version 2.0 of the product
    1,        // string 1 for manufacturer
    2,        // string 2 for product
    0,        // string 3 for serial number
    1        // number of configurations
};

In our get descriptor callback function, here’s where we return this data:

void usb_get_descriptor_callback(uns8 descriptor_type, uns8 descriptor_num,
                                 uns8 **rtn_descriptor_ptr, uns16 *rtn_descriptor_size) {
         uns8 *descriptor_ptr;
    uns16 descriptor_size;
         descriptor_ptr = (uns8 *) 0;    // this means we didn't find it
    switch (descriptor_type) {
        case dt_DEVICE:
            serial_print_str(" Device ");
            descriptor_ptr = (uns8 *)&my_device_descriptor;
            descriptor_size = sizeof(my_device_descriptor);
            break;

Notice that we use temporary variables for the descriptor pointer and its size. It’s only at the end of the function that we copy these into the rtn_descriptor_ptr and rtn_descriptor_size. This saves instructions since the pic instruction set doesn’t make it particularly easy to deal with double-dereferenced data.

Have a look through the rest of the function. You can see how we return descriptors for the device, but also notice how when the host requests the configuration descriptor, it actually gets sent the configuration descriptor, the endpoint descriptors along with any class descriptors as well! The function also returns string descriptors, which are used to identify the device in nice plain language. Note that the string descriptors are in Unicode – a 16 bit value for each character. Luckily for us, in English, all you need to do is at a \0 null to each character. To be fair, almost all USB devices have only English string descriptors.

Once enumeration has finished, it is simply a matter of sending data when we want to indicate that the mouse has moved or a button has been pressed. The trick with all USB transfers is that you need to put the data into a buffer before it is requested. In this case, it is not so much of a problem since the pic will NAK any request for data when the endpoint has not been “primed” (or “armed” – all data loaded into the buffer and the pic USB engine informed that it now has control of the buffer).

The host will ask for data at the interval specified in the descriptors. This does mean that there’s a time gap between when we want to send data and when it actually gets requested. This is the side-effect of a system where the host controls all the transfers. This latency is not going to be noticed for mice or keyboards, but can make a difference for time-critical transfers like MIDI data or even serial data. You can send a bunch of data really quickly – but only so often.

In any case, getting back to the joy mouse, notice that we

clear_bit(intcon2, RBPU);

which turns on the weak pull-up resistors for port B inputs, which we then make inputs by:

make_input(JOY_PORT, UP_PIN);
make_input(JOY_PORT, DOWN_PIN);
make_input(JOY_PORT, LEFT_PIN);
make_input(JOY_PORT, RIGHT_PIN);
make_input(JOY_PORT, CENTER_PIN);

and kick off the whole USB excitement by:

  usb_setup();

Nothing will happen from a USB perspective until we enable the USB module:

usb_enable_module();

This routine allows you to “soft-insert” the device. It can be plugged in, powered and running, but only when you enable the USB serial interface module, will the USB side of things kick into life.

When there has actually been some joystick movement or the select button pressed or released, this data is sent to the PC using the usb_send_data routine:

usb_send_data(1, (uns8 *)&buffer, 3, /*first*/ 0); // ep 1

The first parameter is the endpoint number. In this case, we’re sending data from endpoint 1. We also pass a pointer to the buffer, the size of the buffer, and a helper to indicate whether this transfer is the first one. This is important since USB uses two alternating packet types (DATA0 and DATA1) when sending data so that it knows if one was lost. The PicPack library sets up endpoints so that you normally don’t need to know if you’re sending the first packet or not (the data packet type is set to the DATA1 one on initialisation, so that before the first packet is sent, it is toggled to become DATA0. However, there may be occasions that you need to force the packet data type back to DATA0, in which case, pass 1 for the last parameter.

The JoyMouse implements everything USB-wise that is absolutely required, and nothing that isn’t. You can plug a JoyMouse into both Windows and Linux and it will work perfectly fine. In the next tutorial, we’ll dig a little deeper into the PicPack usb library.

USB part 1 - Introduction

The USB consortium claims that there are about 2 billion USB devices in the world. I think they’re wrong. I think it’s easily ten times that. It is truly one of the few universal success stories for interconnection between devices and computers.

When you have a look at the USB specification however, and its implementation for particular devices, you don’t have to read too much to realise that the specification has an extremely confusing nomenclature and even the standard “class” definitions work in different ways to each other. The other practical aspect that makes USB work difficult is that because everything happens in real time and we’re utilising fundamentally slow devices. So in order to make things happen quickly, everything is set up in advance, and then we’re told about the result after the fact. For example, we don’t get a request for some data, then make a call somehow to make that data available to the host. In fact, what happens is that we make the data available in advance of the host requesting it, and we find out afterwards that it grabbed it.

This means that USB debugging is a pretty tricky business. I don’t mean to dissuade you from having a go – the examples in the PicPack library work should make it pretty easy for you to get up and going pretty quickly, especially if all you need is some sort of serial connection. A lot of work has gone into the USB portion of the PicPack library to get things working and make it easy for you to incorporate USB functionality into your products.

But first, we need to understand a little about USB.

I’m not going to go into a full explanation here – there’s tonnes of information on the web and if you’re even vaguely serious about the topic, grab yourself a copy of Jan Axelson’s USB Complete book. It’s one of the best computer books I’ve ever bought. What I am going to do here is to cover the basics so you have enough information to work with.

First of all, the way that the USB system works entails the PC (or host) making requests, and the devices giving responses. Even when devices have something they need to send to the host, the host has to ask for it first. In this way, all the devices can share a common bus – they only speak when spoken to.

If you’re familiar at all with TCP/IP you’ll know that a device has an IP address (eg 192.168.1.3) and address are allocated (like DCHP) to each device. Given that a PC and its devices is a closed system, addresses start at 1 and go up to 127. This is the device number.

In TCP/IP, and each address is broken up into a number of “ports”. In USB land, this concept of ports is called “endpoints”, Endpoint 0 is a special endpoint; its purpose is (generally) to exchange information about the device itself. Other endpoints have purposes based on the device functionality. An endpoint can receive data from the host (called an OUT endpoint) or send data to the host (called an IN endpoint). These “in” and “out” definitions are named from the perspective of the host.

Endpoint 0 is a bidirectional pipe and must exist for every USB device. There can be up to 16 endpoints, but generally only a couple beyond endpoint 0 are ever actually used.

The USB system, like many systems that are designed by committee, covers many possibilities that are not often used in real life. Each device can have a number of different “configurations” of which the host can chose which one to use. While the information supplied as part of the configuration is important, I can’t think of many devices that have multiple configurations (a mobile phone may well use this if it can be a mass storage device or a modem depending on what is requested).

A given configuration can have a number of “interfaces”. In the USB serial port emulator for example, there is an interface that handles the control of the port itself (ie, baud rate, stop bits etc) and an interface for the actual exchange of RS-232 data. Each of these interfaces are associated with one or more of the endpoints available.

In summary – a device has (usually) one configuration, which has at least one interface, which has associated with it one or more endpoints.

When a USB device is first plugged in, the host and the device go through a process known as “enumeration”. This is how the host finds out about the capabilities of the device and loads whatever drivers are required; the device can also find out how the host wants to handle things. Enumeration takes place using what are called “control transfers”. These are the most complex aspect of anything to do with USB programming. Thankfully, the PicPack library handles the difficult parts of this for you.

Enumeration involves the delivery to the host of data structures that describe what a device is and how it works. These structures are called “descriptors”. You’ll find descriptors for the device as a whole, the configuration(s), interfaces and the end points themselves. Depending on the particular device, there are often extra descriptors that are specific to a particular class of device (eg, a serial port emulation, or a human interface device).

Like most of the PicPack components, pic_usb does the hard work of handling the common cases, leaving you to implement the logic that makes your device different from everything else. In the next tutorial, we’ll look at how the library works.

New PicPack - with USB support - RC1

I've made available the first release candidate for the new version of the PicPack library. It has support for USB pics, including examples of a mouse and a USB serial port.

It needs some tidying up, documenting and no doubt, bug fixes, but I wanted to make it available to the adventurous to have a play with and give me some feedback - much appreciated guys.

Enjoy.

New toys - Maplin Mobile Power Pack

I found a neat device in Maplin the other day. It's meant to be a power pack for recharging things like your phone or your iPod. It's very slim and has a mini-B USB plug which is used for charging the pack and also as an outlet to charge your phone etc. The specifications say that it's 2000mAh, which given the output is 5v, is not bad at all.


But was it really giving 2000mAh at 5V? Well, there's only one way to find out...


Aha! It's 2000mAh at 3.7v.... Just as I thought. Then there's a boost circuit to bump it up to 5v. So, I make it at about 1480mAh at 5v (actually it would be somewhat less due to it not being 100% efficient). Still, that's not a bad little power pack.

It also comes with a very small mains charger - which also has a mini-B USB connector on it.

They're on sale at the moment for £9.99, which is pretty good for a mains 5V supply, and 5V battery source. I bought three, since I knew that their real purpose in life was not in fact to recharge iPods, but really....to power prototyping boards!

See Maplin's website for the N62FX

18. All meshed up – Part 4 – Working in code

Finally, we’ll have a look at how the packet routines are called. The best place to start is the pic_packet.h file, where we find the definitions of the config parameters and status messages. Pull up the packet demo project and open the pic_packet.h file. Here are the things we need to set in our config.h:

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

// pic_packet defines

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


You’ll need as big a transmit queue as possible, but this really depends on how many close nodes you’ll have in the mesh:

#define PKT_TX_QUEUE_SIZE 5

The “Seen List”, is surprisingly, a list of all the packets that we have seen (that are destined for us). The smaller the seen list, the more likely that we’ll think that a packet is new when it’s not, that is, the smaller the seen list, the more likely we’ll see duplicate packets.

#define PKT_SEEN_LIST_SIZE 5

Send Max Tries is the number of times a node will try and send a packet to the destination. The first time is always a direct send, after that, all other tries are routed sends. Of course, higher retry counts mean the packet spends more time taking up space in the TX queue as well.

#define PKT_SEND_MAX_TRIES 4

As we’ll see later, the tick time for packet fun should be about 0.25ms. In the packet example we set a resend delay of 10,000 ticks – or about 2.5 seconds. In real life, this is way too large, but for demo purposes, it allows us to understand how the packet network works. You should set this delay based on the speed of transmission and the possible number of hops in your mesh to the destination node. Remember, even in ideal conditions you need to transmit the packet through three intermediaries to the destination, then the destination node needs to send a packet back in the same way. It should be as small as possible, of course, but the best value will be found by experimentation.

#define PKT_RESEND_TICK_DELAY 10000

The payload is the data that actually makes it to the other end. This delivery system is designed to handle small messages – not large data transfers. 8 bytes might not sound like a lot, but for controlling lights, sending temperature readings, time and date and status checks, it’s really plenty. It’s a stack bigger than what X10 can move around. You can, of course make this as large as your given transceiver can cope with. The protocol we’ll layer on top of the packet network, however, expects a payload size of at least 8.

#define PKT_PAYLOAD_SIZE 8

There are a couple of callbacks that are not enabled in this demo, but can be defined when you want your code to control a little more of what goes on. You can get a call back when a send fails:

// Define if you want to get a callback on send failure
// #define PKT_CALLBACK_ON_SEND_FAILED
// if you define it, you'll need to include this routine in your code:
// void pkt_send_failed_callback(uns16 dest_addr, uns16 pkt_id) {
// }

Or when a send succeeds:

// Define if you want to get a callback on send success
// #define PKT_CALLBACK_ON_SEND_SUCCEEDED
// if you define it, you'll need to include this routine in your code:
// void pkt_send_succeeded_callback(uns16 dest_addr, uns16 pkt_id) {
// }

The packet delivery system expects at this stage to be running on a Nordic nRF2401A or a nRF24L01 chip. You can mix these interchangeably on the same network, but a given node can of course use only use one or the other. Define one of them here:

// define one or other of these:
#define PKT_USE_2401A
//#define PKT_USE_24L01

And lastly, define the level of debug you want. Note that more debug means more serial_print_str statements, which means much larger code size.

// define if you want debug
//#define PKT_DEBUG
// define if you want high amounts of debug
//#define PKT_DEBUG_HIGH

I’m assuming here that you also have #defines in the config.h file for the RF chipset – this has been covered in a previous tutorial.

Unlike other PicPack libraries we’ve used before, there’s no “setup” routine for the packet library. The setup routine normally gets the ports and pins set up correctly as inputs or outputs, ready for communication with the outside world. This library talks to the world via the RF library, so it doesn’t need to set up any ports and pins itself. It does have an “init” routine though, which is used in a library to get any internal data set to the right values and initialise communication with a device so that it’s ready for use.

pkt_init(my_addr, last_pkt);

pkt_init gets the packet delivery system ready for use. It stores the address given as the local node’s address, and also stores the last pkt ID. All the transmit and last seen queues are cleared and everything is set ready for packet reception or delivery.

So what happens when a chunk of RF data actually arrives? Well, assuming we’re using a SparkFun Terminal Development Node (TDN), you’ll remember that we tied the DR1 line to the PortB, pin 0.

set_bit(intcon, INTE); // Enable interrupts on rb0

The default for the port B, pin 0 interrupt is that it is triggered on a rising edge, that is, when the input signal goes from low to high. Perfect for us, so no changes are required here.

In the packet demo, a routine called configure_pkt() does the work of getting the packet delivery system in order, ready for us.


void configure_pkt() {
uns16 my_addr;
uns16 last_pkt;
my_addr = eeprom_read(EE_MY_ADDR_H);
my_addr <<= 8;
my_addr |= eeprom_read(EE_MY_ADDR_L);
last_pkt = eeprom_read(EE_MY_LAST_PKT_ID_H);
last_pkt <<= 8;
last_pkt |= eeprom_read(EE_MY_LAST_PKT_ID_L);
serial_print_str("My addr: ");
serial_print_int_hex_16bit(my_addr);
serial_print_nl();
serial_print_str("Last pkt: ");
serial_print_int_hex_16bit(last_pkt);
serial_print_nl();
pkt_init(my_addr, last_pkt);
}

Essentially, this just pulls the address and last packet ID sent from the eeprom and calls pkt_init. It also prints out the local address and last packet ID – not essential, but very, very handy! Okay, so the packet system is all ready for use, let’s see what happens when the nRF2401A lets us know that a packet has arrived. It does this by taking the IRQ line high, which results in the INT (Port B, pin 0) being triggered. Here’s the interrupt routine:

void interrupt( void ) {
timer_handle_0_isr();
if ( test_bit(intcon, INTF) ) {
if (pkt_receive_level < MAX_PACKET_QUEUE) { // if we're not already processing a receive
if (pkt_receive_level == 0) {
pic_rf_receive(rf_rx_buffer, PKT_PACKET_SIZE);
} else {
pic_rf_receive(rf_rx_buffer2, PKT_PACKET_SIZE);
}
pkt_receive_level++;
} else {
dropped_packet++;
}
clear_bit( intcon, INTF);
} // pkt available interrupt
serial_handle_tx_isr();
serial_handle_rx_isr();
}

The PKT_FLAG_RESEND is used to indicate that you want the system to resend the packet if it doesn’t get through, up to the maximum number of tries you’ve set in your config.h file. This is the usual case. In normal circumstances, it’s only acknowledgements that get sent using PKT_FLAG_NO_RESEND – and these are taken care of automatically.

That’s it! Not much code required to build your own meshed packet network. Next exciting episode, we’ll look at how we can put an application layer on top of this system for the purpose of delivering actual data across the network. We’ll turn a SparkFun TDN into a remote temperature sensor and display the temperature on the base station on an LCD screen.

PicPack 1.2 released

PicPack 1.2 has been released! You'll need this version for the complete packet tutorial.

Many thanks to Peter Lawson in sunny South Australia who helped tirelessly getting the BoostBloader running on a new class of Pic chips. With his help, the BoostBloader now supports 16f88, 16f876a, 16f877a, 18f252, 18f452, 18f2620, and 18f4520. It should be quite easy to add other pics as well, I'll write a tutorial about doing that shortly.

Enjoy!



Version 1.2
-----------

16 May 2008

pic_pack_lib

ds1631.c

Rewrote to use i2c base library

packet.c\.h

General update, documentation, a few logic errors fixed, new callbacks, functions better named

pic_serial.c\.h

Added 16bit hex print routine

Pic_utils.c\.h
Added support for 18f452 for turn_analog_inputs_off()

protocol.h

Tidied up, moved to bit positions to indicate capabilities,
expanded relay and dimmer options

demos

packet

All new meshed packet demo, based on SparkFun Terminal Development
Node (TDN), but will work on your breadboard of course as well. See tutorial on the website

Boostbloader/Screamer

Support for pics that have a write chunk smaller than 16 bytes (older 18f types)

17. All meshed up – Part 3 – Packet chatting

Enough theory. Let’s get our delivery network on the road. Rather than look at the code in this episode (boring), let’s jump straight in and get our nodes chatting. Time enough to see how the code works later.

The packet demo is designed to run on the Sparkfun Terminal Development Nodes (TDNs), which are a very handy piece of gear for testing this all out. They have an RS232 connection on one end, a socket for the nRF2401a / nRF24L01 module on the other and a 16f88 in the middle plus a few leds, which help us know a little of what’s going on. Of course you can run this demo on a breadboard with an nRF module or if you’re feeling adventurous, write some code to support a different RF module all together. There are a bunch out there, do send your code in to include in the library if you get something working. I’m pretty interested in seeing some of the CC2500-based modules going, which are similar in nature to the Nordic ones (but are cheaper and have carrier detection – hooray!) or the modules from the fabulous Futurlec (even cheaper, carrier detection but fascinatingly obscure datasheets – fun!).

For the purpose of this experiment, however, I’ll assume you’ve got a couple of SparkFun TDN (Terminal Development Node) and Nordic modules and a couple of serial ports (USB to serial converters will work fine) or a couple of computers with a serial port each.

The first step, naturally, is to get Boostbloader downloaded onto the 16f88 on each TDN. You’ll need the little mini adapter from SparkFun in order to reach their mini-ICSP connector. See the 5 holes together in the middle of the board? That's the ICSP "socket". Make sure you work out which way round the connector goes – pin 1 is designated by the square pad. The datasheet shows how this is connected, but it’s a pain having to check the datasheet every time you want to use the ICSP connector. Remember: 1 is square.

Once this is done and you’ve confirmed the Boostbloader is doing its thing by downloading a flashing-led test app of your choice (a see the PicPack demo directory if you’re feeling unadventurous), it’s time to look at the packet demo and how it works. Download the “packet” demo into the 16f88 and get ready for some wireless magic.

The 256 bytes of EEPROM in the 16f88 is a great place for a small amount of persistent data. In this example we use it for storing the address of the node, and the ID of the last packet sent.

In the mini terminal mode, there are several commands that we’ll use to control the node. The first is the “w” command, which allows you to set EEPROM bytes at runtime. The first two digits specify the memory location and the third and fourth digits specify the value you want to set it to. “w0523”, for example, sets EEPROM location 0x05 to the value of 0x23 (all values in hex). In our case, we want to set the address of the node and the last sent packet ID like this:

w0000
w0101
w02ff
w03ff

You only need to do this once. The first two memory locations (00 and 01) store the local address; in this case we’re storing a 16 bit value of 0x0001. The third and fourth EEPROM memory locations hold the ID of the last packet we sent. In a more complete example later on, we’ll see how we update this with the last packet ID over time. For the moment, setting it to 0xffff as we do here is fine. This means the first packet this node sends will have an ID of 0x0000, so this is great for understanding what’s going on.

Set your other node to an address of something else other than 0x0001 (I suggest 0x0002), and also set its last sent packet ID to 0xffff.

Right, so now you have two nodes up and running and ready to communicate. One is set to address 0x0001 and the other is set to 0x0002. You can check that these addresses are stored correctly by resetting the TDN (by pressing the reset button) and seeing the serial terminal program come up with something like:

My addr: 01
Last pkt: 65535
 Pkt demo
<17:10:36>

Now it’s time to send a payload from one to the other. Here we’ll send a dummy payload that in future we’ll use to query the temperature on another node. Don’t worry about what’s in the payload for the moment, we’re more interested in seeing a packet make it’s way to the destination and get an acknowledgement back. Go to the terminal connected to your 0x0001 node and type:

s02<enter>

This sets the “send to” (destination) address to 0x0002. This demo is slightly slack by only allowing 8 bit address entry, but I’m sure this won’t bother you with only two nodes right now. This setting of the send address needs to be done whenever you boot.

Now, let’s send a packet. Type:

x<enter>

This should send a packet to the specified destination address (0x0002). Since we only have two nodes right now and hopefully they’re quite close, you should find that node 0x0002 responds pretty much immediately. This is what you should see on the node 0x0001 terminal:

>s02
S: 2
>x
>Send...<Snd good to 0002 id 0000>  RX:5

And here’s what you should see on the node 0x0002 terminal:

 <<s: 01p: 06 01 FF FF FF FF BF FF >>  RX:3 

So what’s the deal about these numbers? The demo prints out the return values from the functions that get called. We won’t worry about the functions themselves and what they do right now, but we will look at these values, since they reveal the inner workings of the system. Have a look at pic_packet.h for the #define definitions.

RX:3 - A value of ‘3’ on reception means “PKT_STATUS_PKT_IS_FOR_ME”. It’s a real packet and we haven’t seen it before. We can happily act on this packet. In this demo our “acting” on this packet just involves printing it out to see what we’ve got. You can see the packet was sent from a source address of 01, and had a payload of 06 01 FF FF FF FF BF FF.

RX: 5 - A value of ‘5’ on reception means “PKT_STATUS_PKT_IS_FACK_FOR_ME”. It means we’ve received an acknowledgement and found that it is for a packet we’ve sent. That is, we have successfully confirmed delivery of a packet. Woohoo! We’ve just sent our first packet.

If everything is correct and there are no transmission problems, these are pretty much the only values that you’re going to see with a two-node mesh. Not much of a mesh, really. However, we can simulate what happens in the mesh to help us understand how it works in this simple case. Do note that in this example, the delay between retries has been set deliberately, excruciatingly large so that you can clearly see what’s going on.

Set the sending (destination) address of the 0x0001 node to a node that you know is not on your network (eg, 0x0003) by typing:

s03<enter>

Now from the 0x0001 node try and send a packet to node addressed 0x0003 by typing:

x<enter>

Now, remember from our previous tutorial that we will first try and send to node 0x0003 directly (a “direct send”). You’ll see that node 0x0002 reports a status of PKT_STATUS_DIRECT_SEND (6). This means that it has received the packet, but will not rebroadcast it since the originating node tried to send it directly (no routing).

Since the first packet never got to its destination (an acknowledgement was not received), the node will try to send it again. On the next try however, our node 0x0001 will try and route the packet via anyone that’s listening (a “routed send”). In this simple case, we know that node 0x0002 is listening. Node 0x0002 should respond with PKT_STATUS_NEED_TO_REBROADCAST (9). So 0x0002 rebroadcasts to try and get it to 0x0003 (of course, it doesn’t know that 0x0003 doesn’t exist on our little network).

Now here’s where it gets a bit tricky, but once you’ve got your head around this, you’ll be well on your way to understanding how this all works.

To recap up to this point:

0x0001 has tried to send directly to 0x0003
0x0002 saw it and printed PKT_STATUS_DIRECT_SEND (6) showing that it decided to ignore the packet (it saw that it’s a direct send not destined for itself)
…in time…
0x0001 tried again, this time routing via anyone that’s listening (not a direct send)
0x0002 saw it and responded PKT_STATUS_NEED_TO_REBROADCAST (9), and rebroadcast the packet out to anyone listening (hopefully it can reach 0x0003, or at least someone closer to 0x0003)

Now at this point, 0x0001 should receive this rebroadcast. But it’s the one that sent the packet in the first place! So it should respond with PKT_STATUS_I_AM_SENDER (2) and ignore the packet.

Finally, 0x0001 will give up, printing out

<Send failed to 0003 id 0001>

to show that we’ve had a failed send.

This confirms that 0x0002 has tried to pass the packet on deeper into the mesh, and also confirmed that 0x0001 is happily ignoring this bounce-back. Of course since our imaginary 0x0003 never received the packet, our originating node will try once more. Our plucky little node 0x0002 tries again to forward on into the mesh, but alas, the packet never makes it. If you have no friends, they’re not going to answer, not matter how many times you call.

Clear this is all complete overkill for two nodes (although you do get retries and acknowledged delivery for free). Once you get three or more nodes where some nodes can’t see all the others, I hope you’ll see how powerful this can be for delivering short messages. Next time we’ll have a look at the calls into this library and how they work, finishing our series on the provided packet delivery system works – then later we’ll also cover some implementations of the system, in the first instance delivering temperature data from different nodes.

16. All meshed up - Part 2 - Packet network basics

Now we’ve worked out how to get a chunk of data from one place to another without any wires, it’s time to look at how the PicPack packet network actually works. In the next enthralling instalment we’ll get the actual packet network up and running in code.

Okay, so here’s the problem I was trying to solve. How do you get a message to a device on the other side of the house, reliably? The distance (with brick walls) is going to be too great for a given, cheap, RF transceiver to reach, but we’re pretty much guaranteed to have devices in between the two – so could we use those as relays? It would be pretty nice not to have to configure the network (ie, direct exactly how the network gets messages from one place to another) - since the network may change and configuration of little hardware devices is difficult from the other side of the planet.

So what I wanted was effectively a meshed packet network. But there’s a catch. We want to do this as cheaply as possible, not using devices with large chunks of memory, but on something like a 16f88 – 4k instructions, 384 bytes of RAM. Could it be done?

The trick here is working out a way to discover network routes automatically in small amounts of code. My first thought was a method by which nodes could ask their neighbours if they knew about the node we wanted to talk to. And then if they didn’t know, they could ask their neighbours. Once we had the route, we could store it. Great, so now we’re storing a routing table. And how long do we store it before we throw it out and have to rediscover again when something has changed? Timewise, by the time we’ve asked all our neighbours to ask their neigbours about the node I want to send to, collected the responses, sorted them, then sent the message, well, this is all getting complicated and memory intensive, along with the next ice age having kicked off.

Well, it’s all about tradeoffs, as we know from other tutorials. So here’s our strategy. If our destination node is within range, we’ll send it directly. It then sends an acknowledgement so we know it arrived safely. If it’s not within range, we get everyone that can hear us to broadcast the message. Hopefully, it will be in range of them, and we’ll get an acknowledgement back. In fact, we’ll allow up to three “hops” of rebroadcasts before giving up.

This does mean we’re going to have a lot of traffic bouncing around at times. We minimise this by the three hop rule and also by checking to see we haven’t seen this packet recently (if we have, just ignore it). It will also mean that a packet may find its way by several means to the destination address. We cope with this by sending an acknowledge back for the first one, and then acting on the packet. If another copy of the same packet arrives – we realise that the packet has already been received and don’t act on the packet, but we do send another acknowledge back, since this one may have come a second time because the originator didn’t get our first acknowledge.

It all sounds a little complex, but in use you just set your local address and start sending out packets. It’s a good project to show how the various parts of the PicPack we’ve seen so far come together. And despite all this complexity, we have a meshed packet RF network in 3k of code. Not bad. That leaves you with at least another 1k of code to actually do something with those packets.

Next time we'll have a look at the code that gets the packet network up and running.

15. All meshed up - Part 1 - RF comms

I mean, really. Who doesn’t want to create their own wireless, meshed packet delivery network? This stuff is just, like, cool.

This project all began when my elderly but still energetic parents rebuilt their house and decided in a moment of weakness to allow me to design the lighting and control systems. So naturally I had all the light switches installed as X10 controllers and all the lights themselves connected via X10 lamp dimmers. Then a bunch of X10 PIR sensors send movement information to the house controller PC via a third party receiver. Then as you walk through the house the lights come on when you’re in the room and the controller PC also knows which lights are on.

So it goes in theory. In practice, every now and then an X10 lamp dimmer dies which is a total pain – they’re mounted in the wall cavities after all. Sometimes the modules forget their settings, and the X10 protocol doesn’t actually make it all the way through the house, despite adding an expensive module that’s meant to relay and amplify the signals. And with me living in London and them in Australia, it’s all a little difficult when it goes wrong. Still, it’s cool being able to control their lights from the other side of the planet.

My plan was to replace the unreliable X10 controllers and lamp dimmers with RF modules. Of course you could never guarantee that the modules could talk to each other directly, my parents have internal brick walls that really wreak havoc with anything wireless. So the network would need to be resilient to this type of problem.

And as soon as you start creating something like this, you start to see lots of ways it could be used. Temperature and humidity sensors, real time clock information, movement sensors… you name it!

I did look at Zigbee of course, and that’s great in theory, but the most popular modules require a star network with controllers and don’t actually do proper meshing. Besides, I thought it would be fun to design my own.

I started out with the Sparkfun nrf2041a modules and a couple of their Serial Development Node (version 1). These are designed to work together and the SDNs provide translation of signals to RS232, a 16f88 pic on board and spot to plug a 2401a module straight into. Perfect!

The first chunk of work was to get the actual Nordic transceiver nrf2401a chips working and chatting away to each other. Let’s look at that first, and then in the next article, look at how the packet delivery network works.

The PicPack library supports both the nRF2401A and the nRFL01 chips, seamlessly and simultaneously on the same network. We’ll have a look at the 2401a here. Open up the packet_demo project.

Naturally the first thing you need to define the ports and pins used to communicate with the 2401a in your config.h file:

// - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// pic_rf include
// - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
// For SFE_TDN_V1 board
#define rf_ce_port PORTA
#define rf_ce_pin  6
#define rf_dr1_port PORTA
#define rf_dr1_pin  3
#define rf_cs_port PORTA
#define rf_cs_pin 0
#define rf_data_port PORTA
#define rf_data_pin 2
#define rf_clk1_port PORTA
#define rf_clk1_pin 1

These are the ports and pins used by the SparkFun Terminal Development Node modules. There is, however, a catch. In their ultimate wisdom, the SparkFun guys (bless ‘em) wired the DR1 output from the 2401a to port a, pin 3. The DR1 “Data Ready 1” output goes high when a packet has been received. Now, this doesn’t seem like much of an issues, except that we’d really like to interrupt whatever the pic is doing at the time when a packet arrives. All this polling to find out what’s going on stuff is just silly. So you’ll need to wire the porta pin 3 to port b pin 0. It’s easy, you just need to install a “link” in the alternative connector on the board. You can see from the picture that it's the second and fourth holes on the second row back that need linking.

So, the first thing you need to do, as usual, is run the setup routine that’s part of the library to set the ports and pins to be the right inputs or outputs:

pic_rf_setup();

Then, as usual, you’ll need to actually initialise the hardware itself for use. The trickiest part about using the 2401a is that it pretty much sits there like a dead duck until you get the configuration right, and then it springs into life. Until then, you really have no idea what you’ve got wrong. PicPack provides two routines for configuring your 2401a. We’ll look at the more verbose one first. You need to setup the rf_config struct with all the settings you care about – then pass a pointer to it to the pic_rf_init() routine. Let’s work through the config.

rf_config my_config;

The Nordic transceivers can receive on two (or more) channels at once. We don’t care about this for our purposes, so we set the payload size (or width, in bits) to zero.

rf_config my_config;

For our primary channel, we do care – it has to be the same size as (in this case) our packet size.

my_config.payload_width_ch1 = PKT_PACKET_SIZE * 8;

Here we fill in the address for channel 2, although it’s actually not important want you do or don’t do with these values:

my_config.address_ch2[0] = 0xf0;
my_config.address_ch2[1] = 0xf0;
my_config.address_ch2[2] = 0xf0;
my_config.address_ch2[3] = 0xf0;
my_config.address_ch2[4] = 0xf0;

When it comes to the channel 1 address, we do care. Here’s the default address for Nordic chips, and in this case, we’re using address 0xE7E7E7. We specify two more bytes than we need just for completeness.

my_config.address_ch1[0] = 0b11100111; // addr starts here
my_config.address_ch1[1] = 0b11100111;
my_config.address_ch1[2] = 0b11100111;
my_config.address_ch1[3] = 0b11100111; // only used three but fill
my_config.address_ch1[4] = 0b11100111; // ...for the fun of it

In order to tell the chip we’re using 3 byte addresses, we set the address_width to 24 (8*3).

my_config.address_width = 24; // (6 bits valid)

We want to have a 16 bit CRC check on the packet. This is how the chip tells a real packet from noise in the air. At 1 Mbps you’d be amazed how often noise turns out to give correct CRC values with 8 bit CRCs. 16 bit CRC is much better, but is still not 100%. That’s why the PicPack library has its own check byte as well.

set_bit(my_config.options, OP_LONG_CRC);

Now we enable having a CRC as part of the transmission:

set_bit(my_config.options, OP_ENABLE_CRC);

Turn off the second channel:

clear_bit(my_config.options, OP_ENABLE_CH2);

Shockburst is the name Nordic gives for clocking your packet in to the chip and then sending it out on-air at the correct bit rate in one go.

set_bit(my_config.options, OP_ENABLE_SHOCKBURST);

You can turn this bit off to enable 250kbps rate, which does give slightly longer range. 1 Mbps gives you less chance of packet collisions, which is why I recommend it (as well as giving compatibility with the nrf24L01 chip).

set_bit(my_config.options, OP_ENABLE_1_MBPS);

On the SparkFun boards, they’re wired up with a 16Mhz crystal, which is why we choose that option here:

// (3 bits valid) -> 16Mhz
my_config.crystal = 0b011;

You have two bits (4 levels) of output power:

// (2 bits valid) 11 -> max power!
my_config.output_power = 0b11;

We can choose any of a gazillion channels, and the Nordic chip can hop around then quite quickly. Still, at this stage of the library, we pick one at the start and leave it at that. I thought it would be good to look at a frequency hopping algorithm, but this does get hard when you have a meshed network. Fine when it’s one on one, but when there’s three or more units, how does everyone keep in sync? If you have any ideas, I’d love to hear them. In the mean time:

my_config.channel = 2;     // (7 bits valid)

Last but not least, we put the chip into the receive mode:

set_bit(my_config.options, OP_ENABLE_RECEIVE);

And then with a little flourish, call the routine that takes this information and passed it to the 2401a to get it initialised.

pic_rf_init(&my_config);

As you can imagine, this programmatic way of telling the Nordic chip what to do is easy to understand and gives you the ability to change things at run time. You’ll also realise that this does take a chunk of code (and hence flash) to do all this. If all you want to do is set the config right at the start and not change it after that (other than the channel) then you can use the quick method:

pic_rf_quick_init("\x00\xa8\xf0\xf0\xf0\xf0\xf0
   \xe7\xe7\xe7\xe7\xe7\x63\x6f\x05",
                  2, 1);

This does exactly what the previous chunk of code did. The ‘2’ parameter is the channel number, and ‘1’ turns the receiver on. To generate this config string, use the nrf2401a_config.pl script in the tools directory. You don’t need to know anything about perl (other than having it installed somewhere!) – just edit the file and run it to get it to spit out the config string.

So how do you go about sending data across the link? Well, here’s how the packet routines do it:

void pkt_send_packet(rf_packet *packet) {
  // +3 for RF address (fixed)
uns8 tx_buffer[PKT_PACKET_SIZE + 3];
uns8 count;
tx_buffer[0] = 0b11100111; // address
tx_buffer[1] = 0b11100111; // address
tx_buffer[2] = 0b11100111; // address
for (count = 0; count < PKT_PACKET_SIZE; count++) {
  tx_buffer[count+3] = packet->a[count];
}
pic_rf_transmit(tx_buffer, PKT_PACKET_SIZE + 3);
}

You can see it’s as simple as whacking the destination address on the start. In the case of the packet network, we’re assuming everyone’s on the same address. Of course, you could use these routines to direct your data to a particular node directly using this addressing.

Receiving data is only slightly more complicated. Since we’ve wired up the DR1 line to the port B, pin 0, it means that we can use an interrupt to detect when the nRF2401a wants to tell us something. This is much better than having to poll for data all the time. The 16f88 has two ways of detecting interrupts, but the one we want is when port B, pin 0 goes high. It’s set up like this:

make_input(PORTB, 0);
set_bit(intcon, INTE);

You’ll need to turn interrupts on as well:

turn_global_ints_on();

In your interrupt() routine you can use something like this:

if ( test_bit(intcon, INTF) ) {
pic_rf_receive(rf_rx_buffer, MY_BUFFER_SIZE);
clear_bit( intcon, INTF);
}

Notice how you test for the interrupt flag being set, respond if necessary, and then clear the flag.

That’s all there is to it. Next tutorial we’ll start looking at how to create a packet network on top of the RF communication we’ve set up here.

14. Of Ports and Pins and SFRs

Up until now, I’ve conveniently skipped over how the PicPack libray handles the read-before-write problem on low to mid range pics, or even what this issue is really all about. It provides a simple solution to a complicated problem, but it bears explaining since you’ll learn how these devices work under the covers.

On Microchip micocontrollers, like many similar devices, allow you to make a given pin either an input, or an output. And you can change this programmatically at runtime as well – handy talking to some devices over a bi-directional data line. There’s some circuitry that can detect if there’s a high or low value present on a pin and also what’s known as a data latch attached to the same pins. The data latch is an output device that holds the value you give it until you change it.

The problem with Microchip pic devices prior to the 18f range is two fold – firstly that the output latch isn’t readable and secondly that all outputs are done at the port (not pin) level. What this means is that when you just want to change one output pin in a port from one value to another, the chip doesn’t know what the current values on the other pins are. Say you’ve set pin 1 to high previously, and now you want to change pin 2 to a high as well. It doesn’t have any record of what the previous value of pin 1 was set to (remember, the data latch isn’t readable) and all outputs to the latch happen at a port level (so it has to write values to all the pins in a port at once). The way the chip gets around this is called “read before write”. It reads the value on all the pins in the port (that is, the actual value present on the physical pins of the pin), then changes the pin you’re interested in, and then writes the whole port back again.

So, in our example, it should read a high value on pin 1 (since this is what the latch is outputting), change the value on pin 2 to high and write them both (and the other pins in the port of course) back to the latch, resulting in pin 1 and pin 2 set to high values.

That all sounds fair enough. It’s clearly a bit of a pain, but surely the output of the latch is the same as what you told it to output, so what’s the problem? Well, in theory that’s perfectly true – but in practice, it doesn’t work like that. Imagine that as part of your circuit you have a capacitor connected between your output pin and ground. The reason why you’d do this isn’t important right now. You can imagine that when you set that pin high, it’s going to take some actual real life time to get to the point of actually reading at a high level. Maybe just microseconds, but still, some time. Now imagine if you set this pin high then, very quickly, set another pin high. Your capacitor laden pin is going to be read and the chip may decide that in fact that pin is low at the moment since the capacitor hasn’t charged to the high level.

Eek! That means that the pin is going to get written back as a low! Disaster! In fact, it doesn’t just happen if you have something capacitor-like attached to the pin, it can happen even with a set of leds and resistors. What to do?

The answer is to simulate the readable latch that we don’t have. The PicPack library provides a “shadow” port that it writes changes to before copying it to the whole port. Of course this results in slightly more code – instead of a one instruction bit-set you end up with three. As I’ve learnt with microcontrollers, everything is a compromise. Presumably Microchip saved some money by not having a readable latch in these devices, but it costs you an extra couple of instructions to guarantee the value on an individual pin. The 18f devices (and onwards) have a readable and writable data latch that makes this problem redundant, but without needing to make any chances, the PicPack library makes your code transparently portable between these devices even though 16f devices don’t have a readable data latch.

At the end of the day you can use the PicPack routines without having to worry about what’s happening under the covers – and this makes your code more portable and readable as well. It’s worth explaining how PicPack achieves this without taking too many instructions, and even without having to tell it how many ports your device has.

Open up the include file for the chip you’re working on at the moment. This example uses the 16f88, which for me, is located at c:\Program Files\SourceBoost\include\pic16f88.h.

Pics have a memory space for RAM which includes clumps of “magic” memory locations that do things when you read to them or write to them. These locations are called Special Function Registers or SFRs. You write to a port, for example, by writing a value to a SFR that results in the port being changed.

BoostC include files define the address of these SFRs, and then define a variable that is located at that address. That means that when you set that variable, you are writing data to that memory address – which means writing data to the SFR which causes the chip to do something.

I’m being long-winded about this because it is actually quite confusing until you get your head around it.

Have a look at the include file. You’ll find the definition of our port locations:

#define PORTA                 0x0005
#define PORTB                 0x0006

Notice that the PORTA and PORTB are in capitals. Later on, you’ll see in the same include file:

volatile char porta                  @PORTA;
volatile char portb                  @PORTB;

So, here we define two global variables, porta and portb that happen to reside at the right place that if you read or write to the variable, you end up reading or write the similarly named SFR. Do you see how the variables use lower case letters, but the actual address is in capitals? Thankfully SourceBoost provide these pre-made include files for just about every pic out there. They’re choice of capital vs lower case is somewhat in contrast to other pic compilers, and sometimes the variable names and address locations aren’t even the same as what is in the data sheet for the same pic, just by way of warning. As I’ve said, portability is relative these days, even if we’re all talking C.

Now, all the “bits” of the SFRs are defined in here as well – so you can, for example, set the timer 0 interrupt enable bit of the intcon SFR by using:

intcon.TMR0IE = 1;

This is bit setting notation is handy for single pins, but I prefer the more portable macro that SourceBoost provides:

set_bit(intcon, TMR0IE);

It results in the same code generated, but it means you could port to a different compiler (or microcontroller vendor if need be) more easily.

Now, you can see here we’re setting bit 5 of the intcon variable (actually memory location 0x000b). This is all okay, since SFRs beyond ports don’t have real-life outputs and hence don’t suffer from the read-before-write problem.

In order to make sure we generate as little code as possible, PicPack sets up an array at the same location as the port and tris SFRs in the pic_utils.h file:

volatile uns8 port_array[NUMBER_PORTS] @PORTA;
volatile uns8 tris_array[NUMBER_PORTS] @TRISA;

The tris SFRs are used to set whether a pin is an input (1) or an output (0). “Tris” comes from the (copyrighted) word tri-state, since a pin can be an output at high, an output at low, or an input (and high impedance). You’ll see shortly that you don’t even need to worry about this tris malarkey either with the pic_utils library.

These declarations don’t use any memory up – they just make it handy when we’re moving things around. We also declare the shadow array:

extern uns8 port_shadow[NUMBER_PORTS];

The NUMBER_PORTS define has already been calculated for you, earlier in pic_utils.h. The actual useful functions are defined like this:

#define set_pin(port, pin) \
 set_bit(port_shadow[port - PORTA], pin); \
 port_array[port - PORTA] = port_shadow[port - PORTA];

So our set_pin routine just works out which shadow location it needs to change, then copies the whole byte to the port at the right location. Through a great deal of experimentation, I’ve found that this results in the smallest amount of generated code. Even though it looks like we have lots of calculations to do, if we’re using known ports and pins then the compiler does what’s called “constant folding” and all the calculations are done as the code is compiled, not at run time.

So, to set bit 5 of porta, you need to:

set_pin(PORTA, 5);

Note that this is set_pin, and not set_bit, which is the generic macro for manipulating any byte. Set_pin and its friends are only for actions on ports.

This is why you’ll see these sort of #defines that you provide in your config.h:

#define i2c_scl_port PORTC
#define i2c_sda_port PORTC
#define i2c_scl_pin  3
#define i2c_sda_pin  4

You can see the PORTC in capitals here. Once you’ve defined it, you can use i2c_scl_port and i2c_scl_pin and it’s much more readable form the code what you’re talking about. In addition, if you decide to move where the hardware’s connected to, all you need to is update it in one place in the config.h.

The routines that are provided are:

set_pin(port_sfr_addr, pin)
clear_pin(port_sfr_addr, pin)
toggle_pin(port_sfr-addr, pin)

These work exactly as you’d expect, all buffered and safe from read-before-write problems. There are a couple of other handy functions as well:

test_pin(port_sfr_addr, pin)

returns the value of the pin (given it’s an input) allowing you to use the same naming convention for your input ports as well as outputs, eg:

set_pin(PORTA, 1);
if (test_pin(PORTA, 2) {
    // do something
}

You can also change a pin to a given value, eg,

change_pin(port_sfr_addr, pin, value)

like:

#define MY_VALUE 1
  change_pin(PORTA, 2, MY_VALUE);

Now, because of the way code is generated for pics, when you’re using constants for the pin and port values, these #defines give you nice, simple code. When you’re using variables, however, it does end up with more code. At that point, you’re generally better off using these set of functions:

set_pin_var(port_sfr_addr, pin)
clear_pin_var(port_sfr_addr, pin)
toggle_pin_var(port_sfr_addr, pin)
change_pin_var(port_sfr_addr, pin, value)

These functions actually use subroutines instead of inline code, but will result in less code overall if you want to do things like:

uns8 my_pin;
  my_pin = 5;
set_pin_var(PORTA, my_pin);

It’s a subtle difference, and it other environments, (eg programming for PCs) you wouldn’t even bother make the distinction. When every byte counts though, it’s worth making the effort. Of course, this uses another stack level – so you may wish to swap code size for stack level and use the set_pin version nevertheless.

As a general note, I’m pretty sure none of the PicPack library actually uses the set_pin_var family of functions. You’ll probably find that you generally know what pin you need to change well in advance – ie, at compile time.

Finally, these routines also make it easy (and beautifully clear in your code) to make pins either inputs or outputs:

make_input(port_sfr_addr, pin)
make_output(port_sfr_addr, pin)

Again, this allows you to do things like this:

make_output(PORTA, 3);
set_pin(PORTA, 3);

Notice how you don’t even need to know anything about tris bits and it is quite obvious what’s meant to be going on.

By way of summary, for the imaginary rt373 device, define your ports and pins in your config.h like this:

#define tr373_data_port PORTB
#define tr373_data_pin 1
#define tr373_clk_port PORTB
#define tr373_clk_pin 1

And then you can do things like this:

#include “pic_utils.h”
  // setup ports and pins
make_input(tr373_data_port, tr373_data_pin);
make_output(tr373_clk_port, tr373_clk_pin);
  // clock in the first value
clear_pin(tr373_clk_port, tr373_clk_pin);
set_pin(tr373_clk_port, tr373_clk_pin);
my_value = test_pin(tr373_data_port, tr373_data_pin);

…and so on. Easy! Note that while the PicPack library currently doesn’t do anything differently for PIC18 devices, the routines are perfectly portable between the devices without any code or even config changes.

PicPack 1.1 released

The latest version of the PicPack is now released.

This includes more documentation on units, packet demo (tutorial coming soon) and fixes for boostbloader.

Note the function in the packet demo that allows you to press "download" on screamer to update your software without having to hardware-reset the pic. Very neat!

13. It's about time

Following on from our discussions on getting the temperature via i2c, we’ll look at talking to a real time clock chip such as the ds1307. It’s a little more complicated than just getting the temperature – but mostly because there’s just more data available. The way we interact with the chip is very similar.

Now, you might ask why we need a separate chip to look after the time. Why can’t we do it all on our pic with our nifty timer routines? Well, we could. In fact, you can even drive one of the timers from an external crystal designed for handling real time clock type problems. There are, however, some good reasons to keep it all separate.

The first is that the DS1307 has support for battery backup – from a 3v CR2032 or similar, which will run it for 10 years or more. That’s effectively lifetime of the device. So when the device as a whole is powered off, the time will still be current. The other neat aspect is that the DS1307 has a bunch of non volatile RAM slots left over (56 byte to be exact) which we can use for our own nefarious purposes. You could use a pic that has no EEPROM (these are generally a lot cheaper) and store your info in the DS1307. Neat!

In any case, it’s a good example of another external device we can talk to over the i2c buss. You could connect a DS1307 and an LN75 on the same buss and talk to them both using only two wires.

Open up the ds1307 demo project. You’ll notice that in our configure_system routine we have:

rtc_setup();

This in turn calls i2c_setup() to configure port and pins as inputs or outputs as appropriate, ready for communication.

In request_time() we do some simple requests:

minutes = rtc_get_minutes();
hours   = rtc_get_hours();
seconds = rtc_get_seconds();

And print out the result. All pretty easy so far.

It gets a little more tricky in process_rx() which allows us to enter commands like:

sm23

over a serial connection, to set the minutes to 23. The same goes for hours and seconds, using

sh12
and
ss04

If you have a look at ds1307.c, you’ll notice that most of the routines are simply wrappers around the i2c routines. Some add a little bit of value, by masking out bits, but even then, they’re mostly one liners. This brings us to the challenge of programming such limited devices – you’ve really got a choice between stack size and code size. If you use a sub routine only once – then it’s probably worth turning it into a #define or using the inline keyword. That means the code it uses gets placed in the actual location it’s used, and you save a stack position. In 16f88 devices, there are only 8 places in the stack. If you use interrupts then you’ll use 1 when an interrupt is called, even before you call any subroutines. So you need to make sure the stack never exceeds the interrupt level + main program level. If you do exceed this, on the 16f88 it will happily wrap the stack around and you’ll have lost the first stack position. In reality, this normally results in generally strange behaviour. If your program is using interrupts, and sometimes crashes, prints out strange repetitive things to the serial port or seems to jump back to the boostbloader, then you’ve probably got a stack overflow.

You can check to see if this is possible by looking at the BoostC output. It will print out a quiet little warning that you’ve exceeded the maximum stack level if everything happens at once (remember, BoostC knows about the whole program structure at the start, but it can’t know if the interrupt happens at the deepest main program stack position). I keep getting caught in this – strange behaviour, start putting serial_print_str in to find out exactly where it crashes (strange, those routines have worked for ages) - only to discover that there’s a potential stack overflow lurking around.

Sourceboost provides some very handy tools for find out what to do here. If you press the Code button on the tool bar and open out the pane that the code shows in, you can see the entire call tree of the program. This feature is worth the price of Sourceboost alone – you can easily see which tree branches are the longest and start to do something about them.

The Function Info pane shows more information about each subroutine so you can see which ones are chomping up flash memory and RAM. My only wish here is in a future version that the #Global line will be expandable. I’d love to see where the global variables that are taking up RAM are coming from. The assembly tab then gives you a fantastic rundown of what assembly BoostC has come up with for your routines – and gives you the chance to optimise this. Often you have to be quite crafty to reduce your flash and RAM footprint. Sometimes its just tonnes of serial_print_str statements.

So, if you find that the ds1631 demo is too big for your chosen pic, the easiest way to trim it down is to take out or simplify some of the serial_print_str statements. The main program “help” printout would be a good place to start.

12. Which Pic?

If you’re anything like me, you’ll have had a look at the Microchip web site and been completely overwhelmed by the choices available to you. There are literally thousands of types of pics. How do you choose?

To some extent that depends what you want to do with it, but that’s not a very helpful answer. What I think is more important as you get started is using chips that make development easy. Once you’ve got something developed, if you were to actually take it to market, you’d simply choose the cheapest pic that had all the features you needed – and microchip’s web site includes tools for searching based on all sorts of criteria.

I think the essential things are
a) A hardware UART (serial port)
b) Plenty of flash memory for instructions
c) Support for the tools you use - not all hardware programmers support all pics!

If you’re starting out, I think the best pic to get going with is the fabulous 16f88. It’s got 4k words of instructions, which is plenty for experimenting with. It has 256 bytes of EEPROM for settings, 384 bytes of RAM and ports A and B. It runs at 8Mhz using the internal oscillator, giving you 2 mips and can run at 5 mips with a 20Mhz crystal. It can actually run an entire multi-hop meshed RF connection in 4k, but only just.

Once you start to run out of memory, flip over to the 16f876A. This guy has 8k words of instructions and because it has more pins, it has an extra port C. Other than that, it’s pretty similar.

The Sure PicDem2 board has an 18f4520 on board – this baby has 16k words of instructions, 1536 bytes of RAM and can run at 10 mips on a 10Mhz crystal. Sweet! Since I want to use the Olimex boards for experimenting without having to do surface mount, I’m also looking at the 18f2620, which stores 32k words of instructions, 3986 bytes of RAM and 1024 bytes of EEPROM. That’s one seriously pimped up pic. I’ll post when I’ve got the boostloader working on it.

Avoid any pic with 12 bit instructions. Most 16f pics have 14 bit instructions, all 18f guys have 16 bit instructions. I got a bunch of the funky Sure 5x7 LED array displays, with a view to reprogramming the pic on the back, but it turns out to be a 16f57, which BoostC doesn’t support as it has 12 bit instructions.

Printing things out to the serial port chews up instructions like you wouldn’t believe. I think it’s better to have plenty of Flash and not have to spend time cutting your program down just to get it working – once you’ve got it working, that’s the time for optimising.

11. It's hot in here

I think now’s the time to start interacting with other devices – and the one we’re going to start with will tell us what the temperature is. It’s a great opportunity to learn about the i2c protocol, how to interface with other devices and what a decimal place really means.

The chip of our choosing is the LM75, which like most of these sort of devices has many compatible versions from many manufacturers. Grab hold of a copy of the datasheet for this guy. You’ll notice that it’s an 8 pin device. 3 pins define the address of the device – so you can have up to 8 all connected together and yet address them individually. It has two modes, “shutdown” and “normal”, where shutdown mode will put the device into a low power sleep mode, and normal mode will convert the temperature every 100ms. It has two registers that we care about, the config register and the temperature register. The 8 bit config register is, naturally, where the config bits are stored. The 16 bit temperature register is where the temperature gets put every 100ms. The datasheet specifies that you can read the temperature safely at any time without any fear that it will be corrupted by another temperature measurement while you’re actually getting the register sent to you.

The LM75 uses a protocol called “i2c”, although it’s actually I “squared” C and stands for inter intergrated circuit. I write it without the superscript “2”. You might also see referred to as IIC and some chips use it without referring to this name at all. Presumably their lawyers told them to keep quiet. In any case, it’s a very neat way of communicating bi-directionally with different devices with only signal two wires. You can have several devices on the “buss” and distinguish between them. It even has modes where there can be several devices making requests (acting as the “master” in i2c speak), however, personally I think this is reasonably unlikely to happen in practice. The devices you make requests upon, in our case the LM75, are said to be in “slave” mode. They just do as they’re told.

Now, most pics that you’ll come across have some sort of i2c hardware inbuilt. This means they can handle a lot of the hard work themselves. The unfortunate fact is that mostly this is only as a slave and not as a master. To be fair, it’s pretty easy to implement master mode in firmware and you don’t get a lot of advantages out of doing it in hardware (presumably some code space). In any case, looking at the software implementation gives you a chance to understand how the protocol really works.

My intention here is not to cover the i2c protocol itself – there are tonnes of stuff on the net about this, and an excellent article in a recent edition of the excellent Silicon Chip magazine. If you’re into electronics at all, go subscribe to this magazine and support the Australian economy.

What I do want to cover is the use of the picpack library, which as you would expect by now, implements enough of the i2c protocol to get you talking with your LM75 or other i2c device but keeps you protected from any read-before-write problems on 16f devices. Once you have the “temperature” data in your hand, it’s also interesting to see what needs to be done to turn it into something worth displaying – and what that really means.

Open up the lm75 demo in the demos directory and have a look. You’ll see that since the lm75 library is built on top of the i2c library, you need to include the i2c.c file in your project as well. Have a look at the config.h:

// - - - - - - - - - - - - - - - - - - - -
// pic_serial defines
// - - - - - - - - - - - - - - - - - - - -
#define SERIAL_TX_BUFFER_SIZE 16
#define SERIAL_RX_BUFFER_SIZE 4
//#define SERIAL_DEBUG_ON
// - - - - - - - - - - - - - - - - - - - -
// pic i2c defines
// - - - - - - - - - - - - - - - - - - - -
#define i2c_scl_port PORTC
#define i2c_sda_port PORTC
#define i2c_scl_pin  3
#define i2c_sda_pin  4
// - - - - - - - - - - - - - - -
// General platform definitions
// - - - - - - - - - - - - - - -
#define PLATFORM_TYPE SURE_PICDEM_2
#define PLATFORM_CLOCK 12000000

There’s the usual serial defines at the top, followed by the i2c defines. In this case, the project is ready to go on the Sure electronics Picdem 2 board – a great board for getting into 18f chips, LCD and temperature. We’ll talk about prototyping boards a little later on. Don’t forget to set your platform clock correctly, both for the serial communications and also for the delays used by the i2c routines.

Pack in lm75_demo.c, notice that getting the lm75 talking is as simple as:

lm75_setup();
lm75_set_config(LM75_ADDRESS, LM75_NORMAL);

Remember that each lm75 has its own address – the LM75_ADDRESS define is at the top of the file. Make sure you match it correctly to the LM75 address you set on your hardware (the Sure picdem2 board has it set to zero). In lm75_setup() the ports and pins are set up to communicate hardware-wise with the device. lm75_set_config sets the config register to the normal mode (ie, not shutdown). This isn’t really necessary since this is the default state, but hey, it’s a demo and we need to demo something!

The serial communication parts of the program are set up to read the temperature when you press t and , and read the config register when you press c and . Reading the config register is pretty boring, but it does prove it works. Reading
the temperature is a little more interesting.

To read the temperature all you need to do is call:

raw_temp = lm75_get_temp(LM75_ADDRESS); 

The LM75 gives you an 11 bit temperature reading via a 16 bit number. The first 8 bits are the integer part, the following 3 bits are the “decimal” part. So we have three “bits” of decimal resolution. That means each bit represents a value of 0.125. If we take the second set of 8 bit, rotate it left by 5 bits, we’ll end up with those 3 bits right aligned in the byte. That makes it easier to process. So, if the left-most byte is 25, and the right most byte (after shifting) is 3, then the reading is 25.375. Now, that sounds like a pretty precise reading. What I haven’t mentioned until now is that according to the datasheet, the LM75 is accurate +/- 2 degrees C. So our reading could really be anywhere between 23.375 or 27.375. That’s pretty broad – and you start to realise that three bits of accuracy are pretty meaningless – especially when you convert them to base 10.

The lm75_demo.c includes two routines to convert the three bits to one or two base-10 decimal places through essentially a look up table. You can work our how I’ve calculated the values (multiplying 0.125 and then rounding). Note that given that we’ve only got 8 values available to us with three bits, you can’t even accurately represent 1 base-10 decimal place! Still, we’re just displaying the temperature here, so I’m not going to get hung up about it. The PicPack library includes support for the DS1307 chip, which has +/- 0.5 degree accuracy as 12 bit resolution, but we’ll be exploring that a little later.

PIC terminal
  <19:55:57>
Commands:
t - Request temp
c - Request config
t
>Requesting temp.
Raw temp = 0x1AE0
Temp = 26.88
Temp = 26.9
  >

The other aspect that I’ve left out of this demo as an exercise for the viewer is the fact we are blatantly ignoring negative readings. These are represented as twos complement numbers, about which there’s plenty of information on the web. If you come up with some cool routines to process negative numbers, send them in and we’ll include them in the library for everyone to use.

10. On Display

Eventually, you’ll want your microcontroller to communicate with the outside world without needing to be tethered to a computer. LCDs are a great way to do this, and given almost all the character-based LCDs are based on the HD44780 chipset, or at least a compatible protocol, it’s pretty easy to get these guys going. There’s also a great range of sizes – from cute 2 line 8 character displays to 4 line 40 character monsters. As long as its character based as opposed to graphical, the picpack LCD routines will get you up and running quickly.

There are stacks and stacks of example code out on the net on talking to LCDs. Go wild, go crazy – if these routines don’t help you then feel free to grab something else. The nice thing about the picpack routines is that they work the same way as the others, and always protect you from read-before-write problems. The only area I think the LCD routines needs expanding is in the area of setting character generator ram (you can do this already, but it’s a bit clunky) and supporting 4 line 40 character displays. These are really just two 2 line by 40 character displays with an extra chip enable line to select between the two “displays”. I’ve got one of these beasties, so I’ll add support for it eventually.

Some notes on getting LCDs working from the electronics side. Most of them run at 5v. So if you are running your pic at 3.3v, you’ll need a separate 5v rail to run the LCD. Check the datasheet for your LCD (if it has one!), but you will probably be able to connect the data pins directly to the pin despite the expected level difference. Check your pic datasheet to ensure that it has 5v tolerant pins, and that the LCD thinks that a logic “1” is as low as 3.3v.

If you’re running your pic at 5v, this will not be a problem. Either way, you’ll need a resistor to reduce the current for the backlight if your display has one. You can’t just whack 5v in there – check the LCD datasheet, match the resistor to the current required (there are plenty of web sites on the net that will help you calculate the resistor value). Or, at least stick something in there, 330R would be a good start.

Lastly, a trap for new players is the contrast adjustment. You can get everything else right, but if you don’t have a variable contrast adjustment, then you’re not going to get anything visible on the LCD. A 10k variable resistor, tied between ground and Vdd allows you to connect the contrast input (normally labelled as Vee) to a value between 0 and 5v. You’ll need to play around with the adjustment once you get it connected up and something allegedly displayed.

Now, on to the software.

Have a look at the lcd_demo project in the lcd directory. Note that in the config.h, you’ll see not only settings for the serial connection (which we’ll use to control the LCD) and the platform settings (make sure you set your clock frequency correctly) but also port and pin settings for the LCD. LCDs can communicate using an “8bit” mode or a “4bit” mode. Either way, you still need three other pins (RS, RW and E) to communicate, so at a minimum you need 7 pins. Personally, I’d rather have more pins free to do something else. Sparkfun produces a serial backpack that allows connection to LCDs with only one pin – unfortunately, it uses asynchronous serial communication, which means you either have to give up your serial port for debugging, or bit-bang your own serial routines. One day I’ll get around to rewriting the backpack code to take a synchronous serial connection (ie, two pins). It looks like a fun little development platform in any case. In the mean time, you can connect to an LCD directly using 7 pins. Platforms like the Sure PicDem 2 board will also work with the pickpack LCD library, even though they’re wired for 8 bit connections. This will actually free 4 pins for other uses.

Have a look at the configure_system routine. Notice that they only new calls are to:

    lcd_setup();    // Setup the pins (output / input)
    lcd_init ();    // Configure chip ready for display

You’ll find that most of the picpack libraries have a “setup” routine for the device they’re talking to. This always sets up the configured ports and pins for input or output as required. Note that they expect that all pins are digital pins, so you’ll need to call turn_analog_inputs_off() yourself. The “init” routines actually initialise the hardware ready for use. In this case, lcd_init configures the LCD for 4 bit operation, clears the display and puts the cursor in the top left corner.

LCD communication takes place in one of two ways. It accepts either commands (which do something) or data (which put data into either the screen buffer RAM or the character generator RAM). You can see examples of commands in the main() routine:

    lcd_write_command(LCD_CLEAR_DISP);    // clear the display
    lcd_write_command(LCD_RETURN_HOME);    // cursor back to the beginning
    lcd_write_command(LCD_LINE1);    // goto line 1

and sending data as well:

      lcd_write_data_str("PicPack LCD demo");    // print welcome message

Note that the LCD_LINE1 is simply a command to choose the display RAM location – so you can position the cursor further along the line like this:

    // goto line 1, 6th character
    lcd_write_command(LCD_LINE1 + 5); position

If you get the example project onto a pic, you can issue commands in your terminal program to clear the display or return the cursor home by typing the letter c and , or h and , respectively. Pressing 1 and enter or 2 and will move the cursor to the beginning of the first line or second line. Typing t followed by a string (anything you like) followed by will print your string to the LCD.

The program starts by putting a string on the first line of your display to confirm everything is working.

Neat! There’s not much more to it than that, but it does open the pic up to start making its own display and telling the world what’s going on.

9. It's about time

It soon becomes apparent when working with microcontrollers, that things happening at the right time is crucial to things working at all. In this tutorial we explore timers and some tools to help timing events.

Pic microcontrollers generally have at least one timer, and often three. These are generally registers that will clock over once per instruction cycle, and when they flip over (from 0xff to 0x00) they can generate an interrupt. Of course, counting from 0 to 255 (0xff) at 8 Mhz, which is 2 mips, isn’t very long – and you’re not going to get much done in between each interrupt! 256 clocks at 2 mips is 0.000128 seconds – or 128 microseconds. Not very long at all. So to make it easier to timer longer periods, some timers are 16 bit (65536 clocks at 2 mips is 0.032768 seconds) and most timers allow for something called a prescaler. The prescaler in an independent counter that divides down the clock so the timer is incremented less often. They’re always in powers of two, so you’ll see them in ratios like 1 to 2, or 1 to 4, or 1 to 8 and so on. In the case of a 1 to 2 prescaler setting for example, it results in the timer counting up every 2 clock cycles, not every clock cycle. 1 to 4 results in the timer counting up every 4 clock cycles, and so on.

The picpack library contains several tools for manipulating timers and keeping track of timings. The first tool is a set of routines found in the pic_timer library, for setting up and starting timers. The second is the pic_tick library, which makes it easier to work out how much time has elapsed between events. Later on in this series, you’ll see how important these concepts are when we create our own meshed packet RF network. But for now, we’ll just see how these timers work.

Open up the tick_demo project. There’s two parts to using timers with the picpack library. The first is manipulating the timer hardware itself. Have a look at the tick_demo.c file, and the configure_system routine:


#ifdef _PIC18
timer_setup_0(TIMER_16BIT_MODE, TIMER_PRESCALER_OFF, 0xffff - 3000 + 13); // 1ms at 12Mhz
//timer_setup_0(TIMER_16BIT_MODE, TIMER_PRESCALER_1_TO_2, 0xffff - 1500 + 7); // 1ms at 12Mhz
//timer_setup_0(TIMER_16BIT_MODE, TIMER_PRESCALER_1_TO_8, 0xffff - 375 + 1); // 1ms at 12Mhz
#endif


In this example, we are assuming that the clock is running at 12Mhz and we’re using an 18f type of pic, which has a 16 bit mode for timer 0. Knowing that each instruction takes 4 clock cycles, we would expect this to take 3000 clocks. The first (commented out) line takes that stance – the timer counts up from the value given, so we need to subtract the 3000 from 0xffff. Note there’s an extra 13 added due to the number of instructions taken to get everything processed, the fact the sometimes updating the timer actually stops the timer for a few clocks on some chips. Of course, if we were using the prescaler at 1 to 2, we would only need 1500 clocks – and less compensation. At 1 to 8, we only need 625. The timer is going to be more accurate at the lowest prescalar value (ie, off) – since you can compensate very accurately. By the time you’re at 1 to 8, you’ve got a compensation level 8 times as coarse. Still, unless your clock needs to be mission critical (ie, actually running a real time clock), any of these will be close enough (within 10 clocks of the required value, which is 2.5 microseconds. The timer routines adjust where they can – given your value, they will adjust the next timer loop if the pic has been busy while the timer interrupt occurred and didn’t get to it as quickly as it did the last time through the loop. This means that while one timer interrupt might take slightly longer than you expected, the next one will be automatically shortened to compensate. So on average, it’s pretty accurate and errors don’t get compounded.

Now, have a look at the pic 16 version:


#ifdef _PIC16

timer_setup_0(TIMER_8BIT_MODE, TIMER_PRESCALER_1_TO_8, 0xff - 250 - 1); // 1ms at 8Mhz

#endif


Even though the 16f devices don’t have a 16 bit timer 0 (they have 16 bit timer1 and timer2 modules however), we can pass in the same types of parameters as the 18f equivalent routines. If we assume that we have a 16f device running at 8Mhz, then we’re looking for 2000 clock cycles to make up 1 ms. Counting to 255 isn’t going to get us far – so we have to employ the prescaler. And if we set it to 1 to 8, then counting to 250 x 8 = 2000 clock cycles, give or take. Of course, as we know, there’s not much room for correction at the 1 to 8 prescaler level. However, for most cases, this should be accurate enough. When you’re using timers, once you’ve set them up (as above), you’ll need to start them, as in the main routine:


timer_start_0(); // kick that timer off...


You can also use timer_stop_0() to stop the timer temporarily, with 18f devices you can do just that, stop the timer. With 16f devices, the timer doesn’t have an “off”, so we just turn the timer 0 interrupts off instead.

Right, so now we’ve got a 1ms timer, what can we do with it? Well, let’s say we want to do something every second. Or see how long it’s been since a particular event has occurred. This is where the pic_tick library comes into play. Back in tick_demo.c, have a look at the interrupt routine:


void interrupt() {

timer_handle_0_isr();
serial_handle_tx_isr();
serial_handle_rx_isr();

}


There’s the serial handlers, as usual, but also the timer_handle_0_isr() routine – which takes care of timer 0. Note that in your code somewhere, you need to include a routine called timer_0_callback() like this:


void timer_0_callback() {
handle_tick();
}


Every timer 0 interrupt, this routine will get called. In this case, we call handle_tick() – which increments the pic_tick “tick” counter. So we know our “ticks” in this program are every 1ms. The ticks are counted using a 16 bit counter, with some neat routines to help work out timings.

Now, let’s say we want to do something every 1 second. That’s 1,000 ms. Have a look in the main() routine:


test_tick = tick_get_count(); // find out what we're up to
if (tick_calc_diff(tick_marker, test_tick) >= 1000) { // 1000 - it's a second
serial_print_str(" "); // print something out
tick_marker = test_tick; // reset to find next 1000
}


Tick_get_count() gets the current tick value. Then tick_calc_diff is used to calculate how many ticks there are between tick_marker and test_tick. It’s assumed that the first parameter occurs earlier in time than the second. The routine works even if the tick count wraps around 0xffff. If the difference is greater than or equal to 1000, then we print something out, include the tick count we’re up to. Then we reset tick_marker to the current tick count.

Why >= 1000? Why not just == 1000? Well, you can imagine that if the pic is busy doing something else, even if it’s in the main loop, then it may well be that the ticks creep past 1000, in which case, our next print out will happen after it wraps around 0xffff again, obviously not as good as just being out by 1 tick or so. Note when you run this application however, that since our main loop is pretty simple, we almost never miss the 1000 mark and print out our Ting! on time.

Have a go at tick_demo and see how accurate it is for yourself. Make sure you adjust the timer_setup_0 values for your clock speed. You shouldn’t have to adjust the ‘13’ etc, just the main number, eg, 3000 or in the case of the 16f device, 250. If you were running at 20Mhz, on a 16f88, so an 8 bit timer0, can you work out what prescalar and starting value you would need to get pretty close to 1ms?