probable cause of PS3 controller crashes

[labreuer]

overview

Rate mode got enabled, very probably by a set of stray signals from the receiver that looked like a valid PWM pulse (the receiver interrupts were enabled). The rate mode PIDs were zeroed somehow (I recall Chris saying that all of the EEPROM got cleared at some point and he had to restore it), meaning that all pitch and roll control was lost. This best explains both crashes: I don't immediately drop throttle because that's usually a bad thing. That's what happens if everything else fails, and I've been trying too long to get pitch and roll to work before deciding that they aren't working. The distance between the curb and the first parking spot is too small for how I've currently calibrated myself. So what happened was the quad stayed at the same height and attitude.

recommendations

1. hardcode the quad into attitude mode in the basestation branch
2. do not enable receiver interrupts
3. implement code to read and write all of EEPROM, including checksums, which might need to exclude non-configuration values, and then verify the checksum before arming is allowed (in fact, there are additional things to verify before arming via PS3 controller should be allowed, like all sensors detected and battery level above a certain amount)

details

Note that all links are to the 5b9af3d commit to basestation.

FlightCommandProcessor.h

  // Check Mode switch for Acro or Stable
  if (receiverCommand[MODE] > 1500) {
    flightMode = ATTITUDE_FLIGHT_MODE;
  }
  else {
    flightMode = RATE_FLIGHT_MODE;
  }

ATTITUDE mode: the pitch and roll sticks are effectively the target normal vector for the quad
RATE mode: the pitch and roll sticks are the target derivative of the normal vector

Before I joined Ashima, the attitude mode sucked. Rate mode is trickier to fly at first, making it very hard to learn on. A workable attitude mode came into existence shortly before I arrived at the company, which means I'm not as good a pilot as Ian perhaps thinks I am. At least, it wasn't has hard for me as he and Mark are imagining, if their strongest memories are still from trying to learn on rate mode.

Traditionally, the rear left toggle switch on the Spektrum (the one facing in approximately the same direction as the base of the antenna) could be used to switch between these modes. One might be tempted to obliterate rate mode but I am not convinced that is wise: should attitude estimation fail horribly while the quad is at decent height, there is a chance it could be rescued by switching it to rate mode. This probably would have avoided the horrible crash that Mark and I had in the park in Glendale. We don't know for sure, as the black box got ripped off the quad and lost (it is now much harder for it to be ripped off). Then again, I would need to practice switching from attitude mode to rate mode before it could really be considered a rescue option. I'm also not sure about using rate mode with the PS3 sticks.

The problem lies in the rate PIDs used in this code, in FlightControlProcessor.h:

  if (flightMode == ATTITUDE_FLIGHT_MODE) {
    float rollAttitudeCmd  = updatePID((receiverCommand[XAXIS] - receiverZero[XAXIS]) * ATTITUDE_SCALING, kinematicsAngle[XAXIS], &PID[ATTITUDE_XAXIS_PID_IDX]);
    float pitchAttitudeCmd = updatePID((receiverCommand[YAXIS] - receiverZero[YAXIS]) * ATTITUDE_SCALING, -kinematicsAngle[YAXIS], &PID[ATTITUDE_YAXIS_PID_IDX]);
    motorAxisCommandRoll   = updatePID(rollAttitudeCmd, gyroRate[XAXIS], &PID[ATTITUDE_GYRO_XAXIS_PID_IDX]);
    motorAxisCommandPitch  = updatePID(pitchAttitudeCmd, -gyroRate[YAXIS], &PID[ATTITUDE_GYRO_YAXIS_PID_IDX]);
  }
  else {
    motorAxisCommandRoll = updatePID(getReceiverSIData(XAXIS), gyroRate[XAXIS]*0.8, &PID[RATE_XAXIS_PID_IDX]);
    motorAxisCommandPitch = updatePID(getReceiverSIData(YAXIS), -gyroRate[YAXIS]*0.8, &PID[RATE_YAXIS_PID_IDX]);
  }

The rate mode PIDs got zeroed somehow; a is for the two rate mode PIDs and b is for the four attitude mode PIDs, plus windup guard (rate mode has no PIDs for accel; it only uses gyro):

> a
0.00,0.00,0.00,0.00,0.00,0.00,\r\n
P    I    D                    RATE_XAXIS_PID_IDX
               P    I    D     RATE_YAXIS_PID_IDX
> b
4.00,0.00,0.00,4.00,0.00,0.00,100.00,0.00,-300.00,100.00,0.00,-300.00,1000.00\r\n
P    I    D                                                         ATTITUDE_XAXIS_PID_IDX
               P    I    D                                          ATTITUDE_YAXIS_PID_IDX
                              P      I     D                        ATTITUDE_GYRO_XAXIS_PID_IDX
                                                  P      I     D    ATTITUDE_GYRO_YAXIS_PID_IDX

(see SerialCom.h for the code that dumps this out)

The mode (attitude vs. rate) is set by the MODE channel on the receiver; note that the receiver is still enabled as of 5b9af3d in AeroQuad.ino:

  initializeReceiver(LASTCHANNEL);

It is in that function where the interrupts get enabled (Receiver_MEGA.h):

void initializeReceiver(int nbChannel = 6) {

  initializeReceiverParam(nbChannel);

  DDRK = 0;
  PORTK = 0;
  PCMSK2 |=(1<<lastReceiverChannel)-1;
  PCICR |= 0x1 << 2;

  for (byte channel = XAXIS; channel < lastReceiverChannel; channel++)
    pinData[receiverPin[channel]].edge = FALLING_EDGE;
}

[eelsirhc]

Stray Signals

Have any stray signals been seen during any other flight? The EEPROM PIDs are zeroed out (I have no idea what they should be), and the interrupts are still enabled. However, the raw data values are never read 5b9af3d in Receiver.h. The quad is set into attitude mode each time the serial command packet is read SerialControl.h and there are no other occurrences of it being set otherwise (except for the alleged spurious signal).

For the spurious signal event to be plausible the following happened:

1. The transmitter stopped transmitting, but the receiver didn't know it.
2. A coherent PWM signal was read on the Spektrum receiver which appeared during the flight.
3. The raw signal then sets the MODE channel < 1500

"1" implies the serial code was never called, else bad packets would be registered. "2" that the randomness of the Spektrum is extremely high yet the correct pattern only occurred this once (has the Quad ever self armed?). "3" requires that the code is implicitly setting receiverCommand[MODE] somewhere (given the code quality, I would accept this possibility).

Checksumming

Checksumming the EEPROM is difficult, but not necessarily impossible. Would you still allow configuration over the Serial port, requiring Ian's incremental ones-complement math to update the checksum each time? If not, why not just hard code the values in source and remove the EEPROM code? Running a check-routine is far simpler and allows `looser' configuration settings (e.g. rate mode PIDS !=0 , battery > 11.0v, gravity vector is "down"...)