1. 1 Oregon State University
Support for learning while debugging in a distributed visual programming language
Laxmi Ganesan 1, Christopher Scaffidi 1 , Andrew Dove 2
1 Oregon State University
2 National Instruments

2. Health scientists have often requested help with creating cloud-mobile-web apps
Motivated creation of a new distributed visual programming language, London Tube…
Colleagues have asked for me or my students to implement apps that track physical activity, diet, sleep habits, location, lung function and many other aspects of physical health and behavior. Such a system typically reads data from mobile devices to a database on the cloud, then displays information in a browser. This got me wondering why this is so hard and whether it might be possible for health scientists to create some of these apps for themselves.
Scaffidi, C., Dove, A., and Nabi, T. (2016). LondonTube: Overcoming Hidden Dependencies in Cloud-Mobile-Web Programming, 
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'16),

3. LondonTube example program
The result was LondonTube. Here you see another example program. This one records the accelerometer every 60 seconds, sends it to a database on the cloud, and then plots these data on a chart when requested by a browser.
The three big blue boxes explicitly indicate which code runs on the mobile device, which on the server, and which in the browser. Greenish-yellow lines between boxes explicitly indicate the flow of data between these components. Blue lines within a given box indicate the flow of data among computational elements.
Scaffidi, C., Dove, A., and Nabi, T. (2016). LondonTube: Overcoming Hidden Dependencies in Cloud-Mobile-Web Programming, 
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'16),

4. Time taken in earlier study to implement that simple read-store-display application (minutes)
Traditional tools
LondonTube
CS undergraduate
240 * 9
CS graduate 8
Non-CS undergraduate
N/A ** 
Non-CS graduate
N/A **
We were happy to see that LondonTube users completed the task in less than 10 minutes. They never ran into major problems. They combined together the concepts covered by the tutorial and generally proceeded right to a working program.
On the other hand, even though the other two students had taken an entire course from me, 10 weeks, they not only took the full 4 hours allowed by the study, but they didn't even finish the program. Now, they had each created a class project more complex than this programming task. But when our students go home, I don't think we always know how long they take on homework. These data show how hard projects are with traditional tools.
* = didn't finish
** = didn't try
Scaffidi, C., Dove, A., and Nabi, T. (2016). LondonTube: Overcoming Hidden Dependencies in Cloud-Mobile-Web Programming, 
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'16),

5. Over 2/3 were about how to use LondonTube!
Proportions of questions asked by LondonTube participants during tutorial or task
Over 2/3 were about how to use LondonTube!
Networking, 29%
I know this because I kept track of what questions the students asked during the tutorial and during the task. And when students asked a question, I went over and looked at their code. So I could see what they were struggling to accomplish.
When they asked a question during the tutorial, I answered it. When they asked a question during the task, I didn't tell the answer unless it was just a request for clarification of the programming task. That way, each student was on his or her own for the actual programming.
My graduate student and I categorized the participants’ questions after the study. So here you can see in black that students with LondonTube and students with traditional tools both asked a lot of questions about Choosing—either choosing which LondonTube boxes to drag and drop from the palette, or choosing which API to invoke. But although the LondonTube users asked a few questions about networking, about loops, and about why the language was designed the way it was, they didn't ask any questions about parallel programming. Those with traditional tools did ask a lot about asynchronous code and passing data between machines. Many of these questions would be superfluous in LondonTube: the notation makes these aspects of the code explicit, as we had intended.
Operators
Scaffidi, C., Dove, A., and Nabi, T. (2016). LondonTube: Overcoming Hidden Dependencies in Cloud-Mobile-Web Programming, 
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'16),

6. Learning becomes even harder when debugging a program closer to realistic size!
The result was LondonTube. Here you see another example program. This one records the accelerometer every 60 seconds, sends it to a database on the cloud, and then plots these data on a chart when requested by a browser.
The three big blue boxes explicitly indicate which code runs on the mobile device, which on the server, and which in the browser. Greenish-yellow lines between boxes explicitly indicate the flow of data between these components. Blue lines within a given box indicate the flow of data among computational elements.

7. New features aimed at easing debugging with the objective of easing learning
Trace: Track how often each node has executed
Flag each node with colored indicators
Green (executed), Yellow (not part of running program) Orange (executing), Red (never executed)
Analyze: Check for missing wires
Green (all attached), Yellow (optional is missing), Red (required is missing)
Combine and explain
Show "most bad" color from Trace and Analyze
Provide tooltip to explain in detail

8. Between-subjects evaluation
Participants
8 undergraduates in computer science
Typical of relatively low-experience research assistants
Procedure
Tutorial: 3-hours, 13 tasks, similar to prior work
Complex, very realistic programming task
Read 3 types of data, store and analyze, construct multi-widget GUI
Measurements
Questionnaire on usability and debugging features
Time taken to complete programming task
Record questions asked, look for improvement

9. Usability ratings: 26 of 36 beat baseline
Below: # of treatment ratings that exceeded corresponding control average
Time: 138 vs 76 minutes
(2 from each group finished)

10. Questions asked by participants
Distribution by topic
(open coding)
Treatment vs control
(reduction where hoped)

11. Discussion
Small features that support debugging can aid programmers’ learning of a language
Areas for improvement noted
Further details to user beyond tooltips
Enhancements of colors and iconography
Empirical studies of generalizability to other languages
National Instruments identified commercializable elements within LondonTube
Watch for code and ideas in future SystemLink release

12. Proportions of questions asked by participants during tutorial or task
Networking, 29%
I know this because I kept track of what questions the students asked during the tutorial and during the task. And when students asked a question, I went over and looked at their code. So I could see what they were struggling to accomplish.
When they asked a question during the tutorial, I answered it. When they asked a question during the task, I didn't tell the answer unless it was just a request for clarification of the programming task. That way, each student was on his or her own for the actual programming.
My graduate student and I categorized the participants’ questions after the study. So here you can see in black that students with LondonTube and students with traditional tools both asked a lot of questions about Choosing—either choosing which LondonTube boxes to drag and drop from the palette, or choosing which API to invoke. But although the LondonTube users asked a few questions about networking, about loops, and about why the language was designed the way it was, they didn't ask any questions about parallel programming. Those with traditional tools did ask a lot about asynchronous code and passing data between machines. Many of these questions would be superfluous in LondonTube: the notation makes these aspects of the code explicit, as we had intended.
Scaffidi, C., Dove, A., and Nabi, T. (2016). LondonTube: Overcoming Hidden Dependencies in Cloud-Mobile-Web Programming, 
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'16),