Fixing laggy SSH connections

This is more of a reminder post as I’m sure I’ll be hitting this problem again and posting about it will help me remember the solution.

So if you’re struggling with a laggy SSH connection to your PI, slow typing, slow output to common commands (like dmesg) and even freezes, try adding these 2 things to your /etc/ssh/sshd_config file:

1. UseDNS no
2. IPQoS 0x00

The second one fixed my problem. I have no idea why but after one day of getting frustrated with what seems like it should just work I started trying all suggestions I could find.

I honestly don’t understand how something so basic can fail like this… By basic I mean used by everyone/everything all the time, not simple.

 

Testing rig

I built a custom testing rig out of some plywood sticks and these kind of elastic cords:

There is just enough clearance for 7 inch props and the quad is free to rotate around one axis (Y or roll in the video). I can easily change the free axis as the cords connect to the arms with metal hooks.

The issue I’m debugging now is related to a tendency of the motors to generate different thrust for the same throttle input. I think it’s because the ESCs have different settings on them as I couldn’t find any issue with my code so far. If I send 50% throttle to all motors the quad spins a lot.

 

si4463 FPV #2

In my last post I talked about using the si4463 chip to send video, telemetry and RC data to the quadcopter. I calculated the bandwidth and it seemed that 500kbps video with 5/7 FEC coding should be possible.

I wrote the code, linked everything together and it kind of worked. The video was stable, not many packets lost but the latency was pretty bad. It got up to ~200ms from the current 100-130ms but worse than this, the video was very choppy. I managed to narrow it down to the H264 codec in the raspberry pi: the bitrate you configure in the codec is an average bitrate, per second. Each frame can vary a lot in size as long as the average is preserved (with some allowed over and undershoot). I got I-frames of 12-16KB and P-frames of 400-500 bytes (at 30 FPS). The I-frames took way longer to send than the P-frames and this resulted in a choppy video. I tried to play around with settings – like disabling CABAC and activating CBR but nothing made the bitstream uniform enough.

The final, biggest problem was actually caused by the RF4463F30 module – and it’s the same problem I had months ago when I tried to use them: they introduce a LOT of noise in the power line, enough to cause all the I2C chips on the quad to fail.

I tried all kinds of capacitors to decouple the module and reduced the noise a lot but there seem to always be some capacitive/inductive coupled noise persisting.

In the end I just gave up and went back to the RFM22B chip which just works. For video I went back to the monitor mode/injection system but did change the modulation to CCK, 5.5Mb to hopefully reduce the possibility of interference with other 2.4Ghz RC systems. CCK is spread spectrum similar to DSSS and should be able to coexist with RC systems around.

 

So yeah, FPV through a very constrained channel with a temperamental H264 encoder and a very temperamental transceiver module is not fun…

 

GPIO ramp up/down

While testing the menu system yesterday I notice a strange problem. Sometimes pressing  a button triggers not only the GPIO associated with that line, but adjacent lines as well.

My first thought was ‘wiring problem’ as everything is crammed inside the RC with 30+ jumper wires, so I started shaking and moving wires around but it didn’t seem to change much.

Then I turned to code – I thought I might have a bug – and after close introspection the code seemed correct:

• Set the column GPIO as output
• Set the column GPIO high (+3.3v)
• Read all 4 line GPIOs, see which button is pressed
• Set the column GPIO as input to turn it off
• Repeat

Everything seemed ok so I took out the oscilloscope and probed one of the columns.

Here’s what I got:

I probe 2 column GPIOs here and the trigger is on the green trace. The 2 problems become obvious – first of all the pin goes low too slow so by the time the next pin is high, the previous one still has ~2.3V, enough to register as high. And second – the code lops through this without any delay between pins.

On paper, the code works correctly but not so in real life where there is inductance and bandwidth limits for triggering GPIOs high/low.

 

The fix is trivial. I changed the loop like this:

• Set the column GPIO as output
• Set the column GPIO high
• Wait 2 microseconds for the pin to settle
• Read all 4 line GPIOs
• Set the column GPIO low <— this causes the voltage to drop way quicker than just setting the gpio as input
• Set the column GPIO as input
• Wait 2 microseconds for the pin to settle
• Repeat

 

Here’s the trace on the oscilloscope after the changes:

 

As you can see there is a clear, safe distance between triggering adjacent column GPIOs and (not visible in the trace) the lines are sampled right in the middle of the high interval.

This works perfectly now.

RC – First HW Test

After a few more failed attempts to print the RC case with ABS I finally gave PLA a chance. Ordered some black 1.75 filament from amazon and a few days later I printed the case successfully from the first try. PLA is great, it’s easy to print, it smells like sugar when printing – as opposed to the chemical smell of ABS – and the quality is really good. However it doesn’t like to be sanded. At all! It’s like trying to sand rubber – or more accurately – sugar.

I decided to stop worrying about the finish so much and ordered some plastic primer and white matte paint. In the meantime I finished the RC PCB and made all the connections and did the first real test.

Here it is:

Features:

• 3 axis gimbal for yaw, pitch and roll. It’s a very high quality one with bearings and hall sensors instead of pots
• Motorized linear pot for the throttle. I went for motorized because when changing flight modes I want to have the throttle in the correct position to avoid stopping the motors
• 0.96″ OLED screen for status info, calibration and other things
• 8 ADC channels – 4 for the sticks, 3 for the individual LiPo cells and another one for the Gimbal Pitch pot
• a 4×4 button matrix implemented with pigpio for all the buttons and switches
• 2 rotary encoders for live editing of parameters like PIDS, menu navigation, etc
• 2x 2.4 GHZ wifi diversity for the video feed
• 5.8 GHZ wifi for the phone connection – to send the video feed through
• 433 Mhz 30 dBm link for the RC data
• 2.2Ah 3S LiPo battery for ~5h of continuous use, with charger/balancer port

The screen is connected through i2c1 at 1Mhz together with 2 ADC sensors (ADS1115).
I’m using this library to talk to the screen but noticed that a full display update takes ~20-30ms during which I cannot talk to the ADC sensors. To fix this, I changed the library and implemented partial screen updates. Now I can call screen->displayIncremental(1000) and the class will send incremental lines to the screen for 1000 microseconds (1 millisecond). The overall FPS is the same as with full updates but I get to do other things while the display is being updated. To avoid tearing I also added double buffering to the class and an explicit swap method.

The end result is a 40-45 screen updates per second but each update is split in 12-13 partial uploads with ADC readings in the middle. So I can sample the ADC at ~600Hz which is more than enough for a RC system.

The RC has 13 buttons and 2 rotary encoders requiring a total of 17 GPIO. Since I didn’t have enough I ended up grouping the 13 buttons in a matrix of 4×4 following this tutorial. This allows me to reduce the number of GPIO to 12 (8 for the matrix and 4 for the rotary encoders). I implemented the matrix reading using PIGPIO and added some debounce code to avoid detecting ghost presses/releases. Seems to work great and it’s very fast.

 

Most of the HW is done and it’s a mess of wires. I’m working now on some videos of potting it together, making the connections and the calibration.

The next step is to work on the phone app to receive the video feed, although I think I will give the quad a test – line of sight.

I really want to fly soon.

RC Almost Finished

Here’s the case after my best attempt:

 

It looks… bad. The paint coat is horrible and full of scratches and the screen is too big.

But worst of all, the screen is not bright enough in direct sunlight. Not even close. I don’t have a photo but after brief testing I’d say it’s unusable.

So I’m pretty disappointing with the result – I ended up with a big, heavy RC system that is too dim to be usable for FPV.

I searched for a week for alternative capacitive touch screens, preferable in the 5-7 inches range but found nothing bright enough under 100 euros.

So after a mild diy depression I got an idea that will solve at lease 3 of the issues – cost, bright screen and the RC size: use my Galaxy Note4 phone as the screen.

The setup will look like this:

• The quad will send video through 2.4Ghz, packet injection (a.k.a. wifibroadcast method) and RC stream through 433Mhz
• The RC will receive both video and RC data and relay them to the phone using another 5.8Ghz wifi UDP connection. The phone will decompress the H264 video using OMX (or whatever is available) and display it with telemetry on top.
• The phone will also act like a touchscreen interface to control the RC/Quad

 

Basically this is what most commercial quads (like Mavic) are doing. I’m sure the video link is 2.4GHz due to longer range than 5.8 and better penetration and the connection with the phone is done over a 5.8, low power link.

 

So the next steps are:

• Redesign a smaller case that will accomodate a Raspberry Pi 3, the RC stick and fader + buttons and wifi cards
• Write a quick android application that can connect to the RC and decompress the video stream
• Profit!

RC Case

I finished designing the case that will fit the silkopter RC system.

Here’s how it looks like:

 

Components:

1. Raspberry Pi3
2. Official Raspberry Pi 7″ touchscreen
3. A 3 axis gimbal stick for the yaw/pitch/roll
4. A ADS1115 ADC to sample the sticks and the throttle fader
5. A motorized 10K fader for the throttle
6. A brushed Pololu motor controller. Pololu modules are awesome btw
7. 2 clickable rotary encoders to tune custom parameters like PIDS
8. 2 switches to save/restore the custom parameters
9. 7 push buttons to change flight modes, RTH and other cool things

 

The case is pretty big due to the screen. It’s 22.3 cm tall, 19.5 cm wide and 4.4 cm deep – so I couldn’t print it in one piece on my Prusa I3 printer. So I had to split in in a few pieces, print each one and them glue them together.

After a few failed prints, here’s the end result:

 

 

I’m painting it now with matte black paint and tomorrow I will put all the components together.

The only problem I’m having now is that I broke my touchscreen while fitting it in the case so I need to order another one.

 

 

Gimbal

I ordered these motors and the GLB Mini STorM32 BGC gimbal controller some time ago to make a gimbal for the Raspberry Pi camera. This weekend I managed to model, print and test it.

It weights ~65 grams, is very compact and fits perfectly on my new quad (more on this some other time).

Here’s a timelapse video of the modelling process:

And here’s a video with me mounting it:

And finally the gimbal working:

It works on 6-8.4V (2S), and is very quick to print.

The total cost is ~50e for the controller and motors. In the future I’ll model some extra motor mounts for some more micro gimbal motors.

 

The STL + Design Spark files will be very soon on github.

[Edit] The files are available here:

https://github.com/jeanleflambeur/silkopter/tree/master/printing/gimbal

Raspberry Pi Camera V2 & GPS

Last weekend I did some tests with a UBLOX Neo6 GPS running next to the V2 camera to see if it’s better than the V1 camera.

The GPS had a clear view of half of the sky with a building to one side so not necessarily ideal conditions. The GPS was powered from the Pi with the cable wrapped through a small ferrite bead.

Here is a picture of the setup:

Here is the graph showing the Satelite number, PDOP, Position accuracy and velocity accuracy reported by the GPS:

There is a clear jump when turning on the camera where PAcc goes from ~2 to ~3.2. With the V1 camera this jumped to 30-40.

The jamming indicator showed 15-20% regardless if the camera was off or on. With V1 the indicator jumped to 100% when turning the camera on.

The number of satellites is the same, as is the PDOP and VAcc.

 

The spectrum measured with the SDR was crystal clear even when the antenna was almost touching the camera ribbon cable – which btw is unshielded.

 

So my conclusion is that the V2 interferes with the GPS way less than the V1 and with a bit of (optional) ribbon cable shielding it should be ok for a multirotor.

 

 

Binary Releases

I decided to start making binary releases available for everyone to download.

Here’s the first one – the GS and the brain binaries:

https://bintray.com/jeanleflambeur/silkopter/gs_brain/d779ff73ee330c82acc82171ddfdbb03701da784

I’ll write a new post later today or tomorrow about the install steps and how to get it working.

Note this is not finishes for flying, it’s an experimental release of the new version of Silkopter that can be used to debug sensors and test most of the nodes.