In my last post I talked about using the si4463 chip to send video, telemetry and RC data to the quadcopter. I calculated the bandwidth and it seemed that 500kbps video with 5/7 FEC coding should be possible.
I wrote the code, linked everything together and it kind of worked. The video was stable, not many packets lost but the latency was pretty bad. It got up to ~200ms from the current 100-130ms but worse than this, the video was very choppy. I managed to narrow it down to the H264 codec in the raspberry pi: the bitrate you configure in the codec is an average bitrate, per second. Each frame can vary a lot in size as long as the average is preserved (with some allowed over and undershoot). I got I-frames of 12-16KB and P-frames of 400-500 bytes (at 30 FPS). The I-frames took way longer to send than the P-frames and this resulted in a choppy video. I tried to play around with settings – like disabling CABAC and activating CBR but nothing made the bitstream uniform enough.
The final, biggest problem was actually caused by the RF4463F30 module – and it’s the same problem I had months ago when I tried to use them: they introduce a LOT of noise in the power line, enough to cause all the I2C chips on the quad to fail.
I tried all kinds of capacitors to decouple the module and reduced the noise a lot but there seem to always be some capacitive/inductive coupled noise persisting.
In the end I just gave up and went back to the RFM22B chip which just works. For video I went back to the monitor mode/injection system but did change the modulation to CCK, 5.5Mb to hopefully reduce the possibility of interference with other 2.4Ghz RC systems. CCK is spread spectrum similar to DSSS and should be able to coexist with RC systems around.
So yeah, FPV through a very constrained channel with a temperamental H264 encoder and a very temperamental transceiver module is not fun…
After a few more failed attempts to print the RC case with ABS I finally gave PLA a chance. Ordered some black 1.75 filament from amazon and a few days later I printed the case successfully from the first try. PLA is great, it’s easy to print, it smells like sugar when printing – as opposed to the chemical smell of ABS – and the quality is really good. However it doesn’t like to be sanded. At all! It’s like trying to sand rubber – or more accurately – sugar.
I decided to stop worrying about the finish so much and ordered some plastic primer and white matte paint. In the meantime I finished the RC PCB and made all the connections and did the first real test.
Here it is:
- 3 axis gimbal for yaw, pitch and roll. It’s a very high quality one with bearings and hall sensors instead of pots
- Motorized linear pot for the throttle. I went for motorized because when changing flight modes I want to have the throttle in the correct position to avoid stopping the motors
- 0.96″ OLED screen for status info, calibration and other things
- 8 ADC channels – 4 for the sticks, 3 for the individual LiPo cells and another one for the Gimbal Pitch pot
- a 4×4 button matrix implemented with pigpio for all the buttons and switches
- 2 rotary encoders for live editing of parameters like PIDS, menu navigation, etc
- 2x 2.4 GHZ wifi diversity for the video feed
- 5.8 GHZ wifi for the phone connection – to send the video feed through
- 433 Mhz 30 dBm link for the RC data
- 2.2Ah 3S LiPo battery for ~5h of continuous use, with charger/balancer port
The screen is connected through i2c1 at 1Mhz together with 2 ADC sensors (ADS1115).
I’m using this library to talk to the screen but noticed that a full display update takes ~20-30ms during which I cannot talk to the ADC sensors. To fix this, I changed the library and implemented partial screen updates. Now I can call screen->displayIncremental(1000) and the class will send incremental lines to the screen for 1000 microseconds (1 millisecond). The overall FPS is the same as with full updates but I get to do other things while the display is being updated. To avoid tearing I also added double buffering to the class and an explicit swap method.
The end result is a 40-45 screen updates per second but each update is split in 12-13 partial uploads with ADC readings in the middle. So I can sample the ADC at ~600Hz which is more than enough for a RC system.
The RC has 13 buttons and 2 rotary encoders requiring a total of 17 GPIO. Since I didn’t have enough I ended up grouping the 13 buttons in a matrix of 4×4 following this tutorial. This allows me to reduce the number of GPIO to 12 (8 for the matrix and 4 for the rotary encoders). I implemented the matrix reading using PIGPIO and added some debounce code to avoid detecting ghost presses/releases. Seems to work great and it’s very fast.
Most of the HW is done and it’s a mess of wires. I’m working now on some videos of potting it together, making the connections and the calibration.
The next step is to work on the phone app to receive the video feed, although I think I will give the quad a test – line of sight.
I really want to fly soon.
Here’s the case after my best attempt:
It looks… bad. The paint coat is horrible and full of scratches and the screen is too big.
But worst of all, the screen is not bright enough in direct sunlight. Not even close. I don’t have a photo but after brief testing I’d say it’s unusable.
So I’m pretty disappointing with the result – I ended up with a big, heavy RC system that is too dim to be usable for FPV.
I searched for a week for alternative capacitive touch screens, preferable in the 5-7 inches range but found nothing bright enough under 100 euros.
So after a mild diy depression I got an idea that will solve at lease 3 of the issues – cost, bright screen and the RC size: use my Galaxy Note4 phone as the screen.
The setup will look like this:
- The quad will send video through 2.4Ghz, packet injection (a.k.a. wifibroadcast method) and RC stream through 433Mhz
- The RC will receive both video and RC data and relay them to the phone using another 5.8Ghz wifi UDP connection. The phone will decompress the H264 video using OMX (or whatever is available) and display it with telemetry on top.
- The phone will also act like a touchscreen interface to control the RC/Quad
Basically this is what most commercial quads (like Mavic) are doing. I’m sure the video link is 2.4GHz due to longer range than 5.8 and better penetration and the connection with the phone is done over a 5.8, low power link.
So the next steps are:
- Redesign a smaller case that will accomodate a Raspberry Pi 3, the RC stick and fader + buttons and wifi cards
- Write a quick android application that can connect to the RC and decompress the video stream
I finished designing the case that will fit the silkopter RC system.
Here’s how it looks like:
- Raspberry Pi3
- Official Raspberry Pi 7″ touchscreen
- A 3 axis gimbal stick for the yaw/pitch/roll
- A ADS1115 ADC to sample the sticks and the throttle fader
- A motorized 10K fader for the throttle
- A brushed Pololu motor controller. Pololu modules are awesome btw
- 2 clickable rotary encoders to tune custom parameters like PIDS
- 2 switches to save/restore the custom parameters
- 7 push buttons to change flight modes, RTH and other cool things
The case is pretty big due to the screen. It’s 22.3 cm tall, 19.5 cm wide and 4.4 cm deep – so I couldn’t print it in one piece on my Prusa I3 printer. So I had to split in in a few pieces, print each one and them glue them together.
After a few failed prints, here’s the end result:
I’m painting it now with matte black paint and tomorrow I will put all the components together.
The only problem I’m having now is that I broke my touchscreen while fitting it in the case so I need to order another one.
I ordered these motors and this gimbal controller some time ago to make a gimbal for the Raspberry Pi camera. This weekend I managed to model, print and test it.
It weights ~65 grams, is very compact and fits perfectly on my new quad (more on this some other time).
Here’s a timelapse video of the modelling process:
And here’s a video with me mounting it:
And finally the gimbal working:
It works on 6-8.4V (2S), and is very quick to print.
The total cost is ~50e for the controller and motors. In the future I’ll model some extra motor mounts for some more micro gimbal motors.
The STL + Design Spark files will be very soon on github.
[Edit] The files are available here:
Last weekend I did some tests with a UBLOX Neo6 GPS running next to the V2 camera to see if it’s better than the V1 camera.
The GPS had a clear view of half of the sky with a building to one side so not necessarily ideal conditions. The GPS was powered from the Pi with the cable wrapped through a small ferrite bead.
Here is a picture of the setup:
Here is the graph showing the Satelite number, PDOP, Position accuracy and velocity accuracy reported by the GPS:
There is a clear jump when turning on the camera where PAcc goes from ~2 to ~3.2. With the V1 camera this jumped to 30-40.
The jamming indicator showed 15-20% regardless if the camera was off or on. With V1 the indicator jumped to 100% when turning the camera on.
The number of satellites is the same, as is the PDOP and VAcc.
The spectrum measured with the SDR was crystal clear even when the antenna was almost touching the camera ribbon cable – which btw is unshielded.
So my conclusion is that the V2 interferes with the GPS way less than the V1 and with a bit of (optional) ribbon cable shielding it should be ok for a multirotor.
In the past week I started working on the new Ground Station. The problems I’m trying to fix are:
- I want a dedicated GS, not a laptop. For this purpose I’m trying a raspberry pi with a 7 inch display (the official RPI touchscreen display), together with a custom case, 2 RC sticks and various buttons. The reason is that a laptop is unwieldy, not bright enough in direct sunlight, tends to get busy with other tasks in the worst moments, doesn’t have a touchscreen (at least my current laptop doesn’t)
- I want a nicer interface that focuses on the main purpose of the GS – flying. The current one focuses more on editing the nodes of the quad.
So I started a new project, the GS2 using QT’s QML and a touch driven UI. So far so good, here are some screenshots:
The resolution is 800×480, a bit small TBH but enough for now. The display size is 7″.
So far the experience in QML is great, it takes very little time to build a nice, functional UI. It does require to build proxies for anything that has to be exposed from the C++ side to qml but that’s a good thing as it separates the UI from the logic.
The main things that I still have to figure out are:
- How to create a property sheet in QML. This is used to show the init params and configuration for nodes. This UI is generated in the current GS so there is no need to write custom UI for every node but just a description of the parameters in a json file.
- How to write the node editor in QML. Screen real-estate is limited and the node structure can get complex. One possible solution is to allow node groups. Basically take the IMU and low pass filters ang build a meta-node – a Filtered IMU – and treat that as a new node in the UI. Or group all the Navio sensors together in a new Navio Node and avoid all the current clutter. This is an interesting feature that deserves a few weekends.
The physical part of the GS – the case, sticks, buttons are still to be designed but I’m sure I can come up with something in a few weekends. The main parts will be:
- A raspberry pi to drive the display and run the actual GS
- An AVR to read the sticks, battery voltage and buttons. Could be replaced with another multichannel ADC and the RPI GPIO
- A few analog sticks. I’m thinking to salvage the ones in my TH9x remote or buy a few of these or these
- A RFM22B and 2 TL-WN722 wifi cards for the comms. This will be in a detachable module in the GS, connected by a usb cable so I can mount it higher or on top of my car.
- LEDs, piezo speaker etc
I already have the rpi + display and the first version of the GS working, check out the video here:
All in all – a few months of work. I hope I don’t get demotivated.
After the latest incident and the loss of my quad it became clear that I need reliability. So far I focused on physical reliability – if the quad is sturdy enough to take a light crash, if it’s well balanced, etc. But I kind of ignored the other 2 aspects – software and comms. Well – not ignored, more like I hoped for the best.
So the next issues to fix are:
- Solid comms for the commands (RC) and telemetry. The RFM22B is the perfect candidate for this:
- It’s lightweight and small.
- Very sensitive receiver (-104dbm @125Kbps)
- Powerful TX amp @20dbm
- 64byte FIFO so I can drive it directly from the raspberry pi
- SPI interface, which the RPI has enough
- Good penetration due to the low frequency (433Mhz)
With this setup I should have 5-10Km range, way more than the WIFI video like allows.
The issue will be the comms protocol as the device is half-duplex and the bandwidth is not that much. I made a quick calculation and it should take around 5ms to send an entire 64byte message. With a simple protocol where the GS is the master and the quad the slave, I can allocate 5ms slots to each in turn. This will give me a symmetric 7Kbyte bandwidth, -protocol losses lets say a 5Kbyte bandwidth per second. Should be enough for control and telemetry data.
2. Solid software, especially the GS. I intend to get a RPI3 and display and build a dedicated GS to replace my laptop. For now I will keep my PS4 controller but in the future I will aim to build an entire RC system with integrated touchscreen around the RPI.
I want to run the RPI without an X server and for this I will need a QT version that supports this. Don’t know if it’s possible though.
If this fails, I’ll stop using QT and do all the rendering and UI using something else but this will be quite painful… I’m used to QT and despite all it’s quirks it’s very powerful when doing quick UI work.
What I do know that works in the current setup is:
- The RPI2 is powerful enough to handle a quad with WIFI video streaming. It was using ~16-20% CPU at any moment with around 5 threads so no issues here
- The case I designed is very sturdy. It’s a combination of 10mm carbon fiber tubes for the arms and ABS plastic for everything else.
- Flight time with the current weight of 650grams is around 20-25 min with a 3000mAh multistar LiHV battery. Enough time to enjoy the landscape
- The motors + ESC pair are a nice match. RcTimer 1806 1450Kv and RTFQ 4-in-1 12A ESC seem to handle the load nicely. A bit underpowered due to the props – 7×2.4inches but I plan to move to 8×4.5 for the next quad.
So back to the drawing board for a while.
I took the quad for a spin yesterday and figured out 4 things:
- It flew, quite good. Very stable, video quality was good and the signal was very strong even 4-500 meters away
- I crashed a few times. See video here: https://www.youtube.com/watch?v=AjwiUQltHaE&feature=youtu.be
- Flight time was around 20-25 min with a 3000mAh Multistar LiHV battery. Average amps for hover was around 6-7 which is great. AUW around 630g
- Eventually it flew away in a near-by forest and I couldn’t find it. The reason was a sudden loss of signal which triggered the fail safe routine. The quad started climbing to 50m, went home and then dropped to 5m. Problem was that ‘home’ was a few hundred meters into the near-by forest due GPS interference from the camera….
So it’s gone, have to build another one.
The lesson here is that I have some software issues to iron out and that I really need another signal path for the control. I plan to use a rfm22b for this.
The material loss is not that huge – a raspberry pi, 4 rctimer 1806 motors, a brand new battery and a Navio board. A total of 250 euros plus about 2 weeks of work to print all the pieces, etc.
So the next one will need to fix these issues:
- Rock solid RC link for control
- Solid software without glitches. My GS crashed once in mid-flight (see video) due to the h264 decoder
- More portable GS. A laptop with 2 wifi cards, a ps4 controller and many wires is not a fun way to go to the field and fly
- The raspicam interferes with the GPSsomething fierce. Even after copper shielding the H Acc went from 2m to 50m after starting the camera…
Some months ago I found this awesome blog with research and a usable implementation of rfmon for uni-directional wireless communication. It uses libpcap and a modified firmware for the TP 721N dongle to send encoded video packets and telemetry to the GS. It’s unidirectional and this has some advantages. The biggest advantage for me was that it doesn’t require pairing nor connection handshake. Perfect for a quad!
So I included rfmon in RUDP and renamed the result to RCP (Reliable Comms Protocol…). It’s bidirectional in my case as I need to send control data to the quad as well but the advantages still apply.
Performance is great, I can send 1024×768 video @30 FPS, 2Mbps using around 4-4.5Mbps with very little CPU usage.
The problem I’ve hit is this: the more data I send, the more packets get lost due to interference, etc. Having a fixed retransmit rate – like every packet 3 times – requires a fixed amount of bandwidth but doesn’t scale at all to low-bandwidth situations. So I need a way to use the least amount of bandwidth possible while still ensuring that frames arrive at the other end.
One solution is to add ACK packets to reduce retransmission – which is what RCP does – but this also has an issue. Every confirmation packet keeps the channel busy for a little while – in turn dropping the total bandwidth of the system.
Current solution is to gather many confirmations and send them with low frequency – around MAX_RETRANSMIT_TIME / 2 (currently every 10ms). So fast enough to avoid retransmission, but not as fast as to keep the channel busy unnecessarily. So far this works beautifully.
The brain on the UAV uses RCP setup with a RFMON socket:
auto s = new util::RCP_RFMON_Socket("mon0", 5);//5 is the end-point ID
util::RCP::Socket_Handle handle = m_rcp->add_socket(s);
if (handle >= 0)
So – this code adds a socket to the RCP instance and then instructs all channels to go through this socket. Same for the internal data – which represents the ACKs, pings (for RTT estimation) and connection requests.
The reason for this indirection is to allow different sockets for different channels. For example – use RFMON for unidirectional video streaming and another socket over a 433Mhz radio (like this one) for all other comms and ACKs.
In the near future I’ll try this – sending all channels except video through the RFM22B socket, as this should give me better range & penetration compared to 2.4Ghz.