Expect the unexpected

Computers are perfectly logical and do exactly as they’re told. That’s precisely what makes programming so difficult and unforgiving.

Add in finicky sensors and inconsistent outdoor and real-world situations, and students of CSE 325: Embedded Microprocessor Systems learned the important lesson of dealing with the new and unpredictable that will be key to their success as systems engineers after college.

“The robotics industry, which spans from Nest Labs and iRobot to Google, Tesla Motors and SpaceX, is looking for engineers that have experience in designing complete, complex robotics systems,” said computer science and engineering associate professor Aviral Shrivastava. “They’re looking for engineers who can write software for systems that have imperfect sensors, inaccurate motors, and so on — that’s how real systems are!”

Over the course of the spring semester, 50 computer systems engineering undergraduate students embarked on the newly redesigned CSE 325 embedded computing course to learn how to be such an engineer through building a self-navigating toy car with GPS, LIDAR (a surveying system that uses laser light to calculate distance from objects), and other sensors including a gyroscope, magnetometer and accelerometer.

Shrivastava noticed the previous version of the course was not keeping up with the current state of the industry and the complex systems his students would face once they enter the workforce. Engineers no longer only work on computing systems in isolation — they must know how software interacts with hardware in order to design a complete system.

The course included eight main projects that cumulatively added autonomous features to the car, and four demonstrations to test that the teams’ cars were meeting objectives. The class culminated in a competitive race for teams to show off the cars’ abilities and earn extra credit.

Students documented their journeys with a video submitted at the end of the semester, like in this video where computer systems engineering undergraduate students Jeremy Quade and Tamim Zawahiri show the concepts they applied. Quade and Zawahiri won the final race with an impressive time of one minute and three seconds. See more team videos on Shrivastava's YouTube channel.

It was a wild but valuable ride in students’ engineering education.

Studying real-world, complex interactions as a new approach to computer system engineering

Thus far in their program, students have mostly learned how to program to get the desired results out of a computing system. Once a programmer knows what to tell a computer for it to perform a given function, and how to do it correctly, programming is easy.

However, in this new course, even if students correctly told their robotic cars exactly what to do, they didn’t always behave as expected.

“Components do not always behave the way they are supposed to,” Shrivastava said. “Even if there is no obstacle, the LIDAR shows an obstacle, or the GPS reading suddenly jumps to show a different location, or when you give an output to turn the car 30 degrees and the car only turns 27 degrees. How to still make the system work correctly, even though the parts do not work correctly, is the big challenge.”

Sometimes this is due to the level of accuracy of the equipment or it can be due to working in an uncontrolled environment where interference is common. Dealing with these errors was particularly difficult for students who primarily worked on more predictable systems.

“Throughout the rest of their coursework, students mostly worked with hardware and software that acts the same way every time, such as programming on a personal computer and its peripherals,” Shrivastava said. “This class was completely different as it involved sensing and controlling complex movement in the real world.”

Learning to overcome this problem was the most valuable lesson that students may not yet have learned in their engineering program.

“This was the biggest challenge, but also the greatest success: Exposing students to sensors and experiences out of the normal classroom track like this has a profound effect on their education,” said Edward Andert, a graduate teaching assistant in the course. “This is particularly important because as embedded systems in industry evolve and start to use new sensors so should our classes that teach them.”

Students also thought this unconventional lesson made them stronger engineers.

“Of course anything that can go wrong will go wrong during the testing demonstrations,” said computer systems engineering undergraduate student Syed Ridhwaan. “Being able to think on our feet to figure out what was wrong and fix it developed some major skills.”

An exciting engineering experience

Ridhwaan and computer systems engineering undergraduate student Ahmed Moussa made up one of 25 team pairs in the class. They were both excited about taking on the challenge of the course.

“At the beginning of the semester after I’d read the syllabus I’d become pretty excited about the course,” Ridhwaan said. “I didn’t know the majority of things we were expected to do, and a challenge like that is always fun.”

Though the projects were stressful and frustrating at times, looking back, Moussa liked the experience.

“At the end of the class I’m glad that I took it, especially in the first class offered,” Moussa said. “When you’re doing something for the first time you get to be a lot more creative, not stuck to a path that people already made.”

Overcoming difficulties with ingenuity, perseverance and teamwork

A sense of creativity served students well, as ingenuity was key to being successful in the class.

Even though Shrivastava and the teaching assistants provided reference materials, there was a lot of creative work that needed to be done to make something work on each custom design.

“We gave them most of the building blocks, but each car is built slightly differently, with different wiring and sometimes customized with lights and switches, so the exact code that will get the car to perform the task is different for each team,” Shrivastava said.

Moussa and Ridhwaan pioneered on the fly a successful three-tier car design — with a convenience store cup, a stick, some duct tape and a broken breadboard — that helped prevent errors from interference caused by parts being too close together.

“The project was due at 9 a.m. and at 4 a.m. we were still getting terrible readings from the compass,” Moussa said. “We needed a way to move the compass away from all the metal, so we tried the cup and it worked.”

It was an achievement to be proud of and an ode to innovating with what’s on hand.

“I loved how we were the very first team to use this design, and everyone started to do the same thing. It held up and got us where we needed to go,” Ridhwaan said.

In order to come up with solutions to their hardware and software design problems they encountered, students showed perseverance and dedication. They were often out late at night at the testing site to try to get code working before their testing demonstration. One night before a demonstration Ridhwaan and Moussa stayed all night Old Main lawn, working until they achieved the needed functionality for the test.

After sunrise their work was undone when the car hit a trashcan and knocked all the wiring loose, and the code no longer worked as expected. The team managed to rewire everything, fix up the code, and got it running before the 9 a.m. demonstration.

Students also strengthened their teamwork skills by working with classmates to overcome various design challenges they encountered.

“There was no way any of us could have done the projects alone,” Ridhwaan said. “We were constantly helping each other by sharing possible algorithms, solutions to common problems, and switched batteries and parts when they broke. It was a little community and who knows how many hours we all saved helping each other out.”

It was a team learning experience for everyone involved, as Shrivastava and the teaching assistants also built a car along with the students.

“When we first got the magnetometer people thought it didn’t work and they wouldn’t be able to finish the project, but the teaching assistants got it to work so they knew it did, and we eventually got it too,” Moussa said. “When we were struggling we’d talk about it in the class and Professor Shrivastava would help.”

Learning valuable engineering strategies to advance the future of engineering

Shrivastava thought students had one of the most important aha moments in learning how to build an autonomous system with feedback control.

When learning to drive the car in a figure eight — two joint circles with differing diameters — they first worked by trial and error. Students would set the steering and note the circle’s diameter. After several steering tests to determine what setting would result in a given diameter, they’d choose the two settings that would get the desired diameters.

The next project added feedback control. Instead of manually setting the steering angle they did a few upfront calculations and incorporated sensor feedback to make adjustments for accurate navigation. This way, if the car starts turning faster or slower than expected based on gyroscope or magnetometer readings, the system will continually readjust to stay on a given diameter.

“Once they did this project they really started to understand how feedback control works and why we need that,” Shrivastava said. “You can sense from the sensors how much you are turning at each moment and adjust the turn to the required degree, thus no elaborate calibration is required. This is calibration on-the-fly.”

This was Moussa’s favorite part of the class, even though it was the one he struggled with the most. But he says he learns the most from messing up, so he enjoyed the challenge.

NSF research goals translated to the classroom

The redesigned CSE 325 course is related to the teaching outcomes of Shrivastava’s National Science Foundation (NSF) project, “Scaling the Real-Time Capabilities of Power-train Controller of Automotive Systems.”

“Even though the regular performance of processors is improving, real-time performance — how many real-time tasks can be executed simultaneously — is not,” Shrivastava said. “The reason is that real-time performance of a processor depends on the worst-case execution time. If the worst case execution time of tasks does not reduce, then the real-time capability of a processor does not improve.”

This is especially important in safety-critical systems where a task cannot ever miss a deadline because it would have disastrous consequences, such as in car software that controls braking.

“For my NSF project I propose new processor designs and ways to write software so that their real-time performance can be improved in safety-critical robotic systems,” Shrivastava said.

Traditionally, improving performance of processors and improving the performance of robotics were two separate topics. This project brings both of them together in an end-to-end solution — where improvement in performance of processors translates into improved performance of robots.

Teaching real-time performance is a critical metric to improve safety-critical robotic systems, and with Shrivastava’s input into this course based on his NSF award project research, students are now closer to being able to design the next-generation of computing and robotic systems. They can see how the mechanical, electrical and computing parts come together to make a complete working system.

A foundation to build more advanced systems

For Ridhwaan this was a helpful start to projects he’d like to do in the future.

“For this course I can’t really think of a better project to work on,” Ridhwaan said. I’d like to program drones and more advanced things in the future, but that’s not an ideal project for beginners. With the car project you’re able to cover all the key concepts for this class in a fun and challenging way.”

Moussa is also ready to take his knowledge and apply it to other projects.

“With the knowledge I have and some time on Amazon looking for parts I can build really awesome things,” Moussa said.

Improvements for next year

It’s a traditionally challenging course in the computer systems engineering curriculum, and perhaps even more so this semester as the first of its new design, but students generally did well. A majority of the teams were able to complete the projects successfully, and Shrivastava hopes next semester’s students will be even more successful with a few changes to the course.

He hopes to get more teaching assistants to help with inventory management, and, most importantly, to provide more support for students to successfully complete this complex project.

Shrivastava also plans to replace difficult parts with ones that will be more accurate, and provide a space that does not interfere as much with the sensors to help make the project demonstrations go more smoothly.

“The field where we tested our projects has too many buildings around it so the GPS signal is not very accurate,” Shrivastava said. “Next year we plan to find a more open space where we will get better GPS reception. A space with fewer bushes that aren’t very visible to LIDAR will also reduce LIDAR problems.”

Advice for the next group of students

“The most important tip is that this course will be a lot of work, but the other side of the coin is that this course provides an opportunity for a tremendous amount of hands-on learning that is not easily possible from any other sources,” Shrivastava said. “This class can give the confidence and understanding to build real, complex systems, which will be extremely valuable in the future robot-filled economy.”

Just like in the workplace, using time wisely and testing in the right conditions is key, Andert said.

“Start on the projects early and test in conditions as close to the testing conditions as possible,” Andert advised. “Don’t code the entire LIDAR project the night before the project in the dark! Otherwise it won’t work in the day.”

Ridhwaan’s takeaway is that it’s tough, but a valuable learning experience.

“You’re going to get frustrated, you will learn a lot, and always help your fellow classmates,” Ridhwaan said.

And, ultimately, Moussa thinks so far it might be one of the best experiences computer systems engineering students will get.

“Have fun,” Moussa said. “If you don’t enjoy this class you should rethink your major because this is probably the coolest project you’ll get to work on!”