1. Integration of Prosperity4All components
Webinar

2. UAS Technikum Wien 4 models Overview Click with a head movement
Click with an acceleration sensor
Move the mouse with head movements and click with dwelling
Move the mouse with head movements and click with an external button
UAS Technikum Wien provided 4 models

3. Click with a head movement
Models
Click with a head movement
Click with an acceleration sensor
Move the mouse with head movements and click with dwelling
Move the mouse with head movements and click with an external button
The first two models need scanning to control the program.
Lets briefly explain scanning now

4. Scanning
The selection frame moves automatically from one selection field to the other.
Space, Enter or the left mouse button selects the object.

5. Click with a head movement
1-Button scanning and the head movement
Clicks need big movements or are prone to errors
If you have control over your Head you can
use an external mounted Button
or an Integra Mouse Plus
Click with a head movement
You can mount the Button beside of your head and press at the right time with your head.
We didnt follow the „click with the head movement“ model any further.

6. Models
Click with a head movement
Click with an acceleration sensor
Move the mouse with head movements and click with dwelling
Move the mouse with head movements and click with an external button

7. Click with an acceleration sensor
The acceleration sensor
- uses a USB cable
- costs around 79€
- needs a special driver (no plug and play)
The scenario here is that you have a user that can raise a hand or foot but cannot really control the direction.
So a button would not be feasible, as the client would not really be able to hit the button in a consistent way.
On the left side you can see the model in a diagnostic state.
The left most diagram shows which raw data is delivered from the sensor.
There we can check and see how all the variables behave while we are raising an arm.
On the right hand side you can the programm Euro with 1-Button-Scanning and as the activation level is high enough the actual scanned lesson was started.
There are a some things that did not make us happy:
The cable
The cost
The driver:
The special driver for the sensor is not signed.
In Windows 10 you have to go through a lengthy procedure to install an unsigned driver.

8. API Other “state of the art” wireless solutions? Activity Tracker
Fitness Armbands
Other “state of the art” wireless solutions?
Activity Tracker
API
The next question we asked:
Could we replace the Acceleration sensor with a more state of the art wireless solution?
These armbands (like the Mi Band) are quite cheap.
I was researching for fitness armbands
Most of them don’t have any API at all and if they had, only iOS and Android were supported.
Of the three remaining devices, two where contacted, if they have a way to get the RAW data of the armband.
The reason for the RAW data is that we don’t need the number of steps a person has walked during a day.
We want direct immediate feedback generate a mouse click in a program.
Both answered this question with no.
The third device is the Angel Sensor.

9. Very successful indigogo campaign Contacted the developers Nov. 2015
Fitness Armbands
Very successful
indigogo campaign
Contacted the
developers
Nov. 2015 (
From developers for developers.
There is a big demand for such a device with direct access to all different kinds of RAW data.
The indigogo campaign goal was surpasse by 300%.

10. - Device could not be ordered
Fitness Armbands
- Device could not be ordered
- Data can only be accessed through the cloud -> quite much latency for a mouse click
Project closed in May 2016 and didn‘t deliver
any Devices to its supporters
Problems:
We were not able to order the armband.
You can‘t develop for an alternative input device without having the device.
Simulation can‘t bring up many issues.
Allows access to the the RAW data but only through the cloud and not locally.
This would mean that you get this chain of commands to make a click in your program:
armband->mobile connection-> cloud server-> web app to push the data ->local connection to AsTeRICS
This would mean at least half a second of latency only because of the many different involved layers.
The last updates of the website are from May 2016.
The people in the forum of indigogo are really disappointed.
This ended our expedition into the realms of fitness armbands.

11. Standard gestures work good (fist, wipe)
Myo Armband
Device can be ordered 
Standard gestures work good (fist, wipe)
RAW-Data is sent in real time to the PC
Very nice browser app
It works with the small current inside the muscles
Big plus points:
It can send its data in real time to the pc: very nice browser app that shows all kinds of data.
The gestures work well as long as you use the standard ones like a wipe from right to left or a fist.

12. Myo Armband You can see all kind of data from this browser windows:
8 different sensor datas
Gesture data
The orientation of the armband is symbolized with the cube

13. Needs calibration before each session Nothing to develop
Myo Armband
Costs 199$
Needs calibration before each session
Nothing to develop
Main reason we did not go further:
It is nearly impossible to put on the armband the same way two times in a row.
This means the muscles of your arm will not be on the same position as last time.
Because of that you have to go to a lengthy calibration every single time you use the armband.
We made a prototype implementation of a One-Button Scanning where you could press the buttons
in the program when you made a fist.
It worked quite good just by using the drivers that came with the device.

14. Models
Click with a head movement
Click with an acceleration sensor
Move the mouse with head movements and click with dwelling
Move the mouse with head movements and click with an external button

15. Move the mouse with head movements and click with dwelling
Overview
Move the mouse with head movements and click with dwelling
Move the mouse with head movements and click with an external button
Control the mouse directly and use different forms of selecting the object under the mouse
AsTeRICS fully covers external buttons ->
if we can dwell we also have external buttons covered
Both models control the mouse in the same way.
Next lets briefly explain dwelling.

16. Dwelling
If the mouse stays on an clickable object a circle appears that is slowly filling up.
As soon as the circle is full the system performs a click on that object.

17. All standard webcams supported No driver installation
Head Mouse
The camera searches
for nose and chin
No calibration needed
Robust
All standard webcams supported
No driver installation
Often no extra hardware needed
(Needed hardware is cheap)
No calibration needed: any person that sits in front of the webcam is immediately recognized
Robust against influences: works even if the user carries glasses or if the room is quite dark or very bright
All standard webcams: as long as you can see a webcam picture in your system it should work with AsTeRICS
No driver installation: the webcam is part of the system anyway
Often no hardware is needed: and if it is needed the hardware is cheap

18. Technical Implementation Details: REST
Communication via REST
First usable version in AsTeRICS 2.6
Final version is now in AsTeRICS 2.8
var request:URLRequest =
new URLRequest ('
request.method = URLRequestMethod.GET;
new URLRequest ('
request.method = URLRequestMethod.PUT;
new URLRequest ('
In AsTeRICS 2.3 you had to use a local udp port for communication.
a new way of communication via REST was implemented in p4all.
REST was implemented at UCY
Here you see snippets of code where we access the REST interface.
State with GET is needed to know if AsTeRICS has been started without any errors.
Start with PUT starts the model and captures the camera input to move the mouse.
Pause with PUT is called when we open the option dialog.
While you are changing options you probably dont want to move the mouse.

19. Solved together with UAS Technikum Wien
AsTeRICS was blinking in the Taskbar
AsTeRICS now runs in the System tray
AsTeRICS needs the Java 32bit Runtime
A package with the embedded runtime was delivered and this is now installed.
No more dependencies on system components.
Solved Issues together with UAS Technikum Wien
Things that we needed to solve to make a final product out of this test version.
When we start FlashWords, we also start AsTeRICS .
And because the loading takes place in the background,
it could happen that the taskbar appeared and the AsTeRICS runtime was flashing.
So with version 2.8 AsTeRICS now runs in the system tray.
For us it is very important to deliver the runtime with the program.
We don’t want to change anything on the pc of our customer.
On the one hand if we install our own version of java, we cannot guarantee that all the other programs on the computer are not affected.
On the other hand if any other program installs a different java runtime, we cannot guarantee that our program works will all other possible runtimes.

20. Solved together with UAS Technikum Wien
Setup installs all components of AsTeRICS
- needs time
- needs space
- C++-Setup
- licensing issue
- Starting of AsTeRICS
could take >30sec
AsTeRICS P4AllBuildingsBlocks
- Install only what you need
- Licensing files easy available
- Starts in <15 sec
Solved Issues together with UAS Technikum Wien
All components are installed even though our model does not use any of the
Plugins that need these installation files.
For Instance a C++-Redistribution-Setup is started during the Setup-Process
Even though our model never needs the Plugins that depend on that C++-DLL.
Licensing is very important for us, as we can not take the risk of a law suit here.
We can not use anything that is not free for commercial use.
We have some older developer pcs that needed quite some time to start up the whole program.
If you are watching a “Loading”-screen for 30 sec you may start to think that the program may have crashed.
P4AllBuildingBlocks creates a small subset of AsTeRICS that
Contains only the files that are necessary for your model.
And you get a detailed overview of all the licenses your model needs.

21. Technical Implementation Details: Setup
Setup for Windows 10 proved tricky:
- Batch call of AsTeRICS did not work in Windows 10
- Virtualization of write accesses
Temporary solution:
Extra executable that starts both executables
Final solution:
Installation automatically into the user directory
We had to start AsTeRICS together with FlashWords and in some cases depending on operating system and
Installation path this did not work.
AsTeRICS also needs some write access to the file system.
This access is virtualized by the operating system, if you install the program under the programs folder.
In our first test prototypes I made an extra executable that took care of the starting process.
As we later automatically changed the target installation folder to the user directory, the virtualization didn’t not happen anymore.

22. Broad and deep target system tests are invaluable
Takeaways
Models in AsTeRICS 2.8 can now be started/paused/ stopped with F5/F6/F7
Broad and deep target system tests are invaluable
All software stops working sooner or later.
This is even true in the short development
cycles we have in this project
In AsTeRICS 2.8 you can use F5/F6/F7. Our Models had an extra implementation with F10/F11 to start and stop.
Broad and deep target system tests are invaluable: In terms of operating systems and hardware
There are so many variables you don‘t know and the only way to be sure is to test.
Without Maintenance all software stops working sooner or later:
You need updates for software because of security, stability and features, just to name a few.
But every update carries a risk.
It happened 2 times during the project in the case of FlashWords:
First time in FlashWords was with the change to AsTeRICS 2.8 as the base path to the REST server had changed.
Second time was with the acceleration sensor as it was not recognized under window 10 in the default configuration.