1. Effective Course Evaluation
EvaSys Survey Automation Suite
Key Points
Welcome
Introduce yourself and colleagues
Confirm timings for meeting
Good morning/afternoon and thank you for joining us today. My name is John Atherton and I am the Director of Marketing & Sales for Electric Paper Limited. With me today are my colleagues XXX and XXX who are available to answer any questions.
I am going to start with a presentation about Electric Paper and EvaSys, I then will follow that with a demonstration of the software. The presentation will last about an hour and the demo should take about 15 minutes. With questions, the meeting may take up to 2 hours (wait for any objections).
Philip Wyse
Managing Director

2. Prepare Quality Survey Analyse Capture
EvaSys
Prepare
Quality
Survey
Key Points
EvaSys manages the full evaluation cycle
Automates the process
EvaSys on the other hand has been specifically designed to manage the complete evaluation cycle and focuses on automating many manual and repetitive constraints in the process.
At the end of each semester/term, or whenever you run your evaluations, it is likely that you will go through these steps.
Firstly, preparing what is to be surveyed and when. Generating and distributing surveys. Capturing responses and sharing the results. And finally, monitoring the overall performance or quality of the results.
Analyse
Capture

3. Design questionnaires
EvaSys
Prepare
Import data
Define structure
Design questionnaires
Select survey type:
- Hybrid
- Paper
- Online
Quality
Survey
Key Points
Explain the features for ‘prepare’
EvaSys provides the opportunity to import all your course, user and participant data which in turn allows you to configure an organisational structure in the software.
Paper and online questionnaires can be designed using the web-based editor.
And there are various delivery options to choose from.
Analyse
Capture

4. Prepare Quality Survey Analyse Capture
EvaSys
Prepare
Batch generate surveys
Batch print paper surveys
Batch paper surveys
Schedule online surveys
VLE integration
Mobile device friendly
Quality
Survey
Key Points
Explain the features for ‘survey’
You can then use the batch processing feature to generate and distribute paper and online surveys, meaning that you do not have to manually repeat the process for each survey.
There are also features to schedule tasks for online surveys and the ability to integrate with VLEs and mobile devices.
Analyse
Capture

5. Multiple scanning locations
EvaSys
Prepare
Scan paper surveys
Multiple scanning locations
Outsource scanning
Online submission
Data entry
Data verification
Quality
Survey
Key Points
Explain the features for ‘capture’
The next step is to capture responses. For paper surveys we use image based scanning and an application called the Scanstation.
You can scan questionnaires from a central location at the university, or at multiple sites/campuses, or even outsource the process to Electric Paper.
For online surveys, users simply submit their response and there are also some optional user roles for data entry and verification.
Analyse
Capture

6. Batch download reports
EvaSys
Prepare
Batch reports
Batch download reports
Aggregate responses
Filter responses
Compare responses
Export responses
Interrogate database
Quality
Survey
Key Points
Explain the features for ‘analyse’
As soon as you capture 1 online response or scan 1 paper survey EvaSys will automatically generate a PDF report. You can then use the batch processing features again to distribute the reports to your colleagues.
Using the Report Creator there are then options to aggregate, filter and compare responses.
And, there are various export options and the ability to query the database tables directly.
Analyse
Capture

7. Prepare Quality Survey Analyse Capture
EvaSys
Prepare
Quality
Survey
Set quality targets
Traffic light reporting
Reporting dashboard
Highlight performance
Key Points
Explain the features for ‘quality’
Summarise why EvaSys is different to generic survey tool
Finally, it is possible to set targets on your questions which result in traffic light reporting, including dashboards to highlight overall performance.
You would then repeat this process for each evaluation campaign.
As you can see, due to all these features and processes, EvaSys is not a generic survey tool. It can run any type of survey, but it is focused on automating the evaluation cycle and managing course evaluation data.
Analyse
Capture

8. Prepare Quality Survey Analyse Capture
Key Points
EvaSys cycle
As explained this is the EvaSys course evaluation cycle. I will now take you through the cycle and explain the key features.
Analyse
Capture

9. CSV XML SOAP API Data-bridge
Data import
Student
Records
Database
CSV
XML
SOAP API
Data-bridge
Key Points
Explain SOAP API import
Finally, it is possible to create a ‘live feed’ of data from your existing systems to EvaSys by using the web services SOAP API. This would effectively just push data into EvaSys without the need to export and import. A data-bridge is usually a custom project, developed by our Professional Service team.

10. Paper questionnaires Hybrid Hardcopy with QR code
Each questionnaire bar-coded
Used when for paper & online
Key Points
Describe hybrid & when they might be used
The second type is called Hybrid.
Hybrid questionnaires are the same as Hardcopy in that the questionnaire details and course details are coded into the barcode, making each form unique. However they also include a QR code which can be used to access the survey online.
Participants can simply choose to scan the QR code with their mobile device and then complete the survey online.
The barcode at the bottom of the form contains the same information as the QR code, meaning that EvaSys will not allow duplicate responses from each individual questionnaire.

11. Online questionnaires
Mobile
QR code
Send invitations
Access survey
VLE integration
Key Points
Describe options for accessing the survey
It is also possible to integrate EvaSys online surveys with a VLE such as Moodle or Blackboard.
As a student logs into the VLE using their ID or address, the building block will look up in EvaSys whether there are any open online surveys for the student. If there are, a box will appear on the VLE homepage that says ‘you have surveys to complete’ and it will list them by course name. Once the surveys have been completed the surveys will disappear.
The passcodes works in the same way, so that if a survey is completed through the VLE, then the passcode will be used up and the student will not be able to access the survey via their , and vice versa.
Surveys are also mobile friendly. EvaSys will automatically detect the screen resolution and resize the page to fit the screen, making it easy to complete the survey. An app is not required for this, but it would be possible to integrate into a university app as long as the student is identified.
Finally, as already mentioned, it is possible to access online surveys via QR code.
Survey

12. Online questionnaires
Send invitations
Access survey
Send reminders
Close survey
PDF report
Results
Key Points
Describe options for closing the survey
Once the student completes the survey, EvaSys will display links to any other surveys that the student still needs to complete. You can see on screen that the green tick is for the survey that was just completed and that there are two other surveys available to take. This is a useful feature to help encourage continual participation for online surveys.
As the survey closes EvaSys will send an with the results in a PDF format attached to the lecturer.
Surveys reminder

13. Capturing & feedback Key Points Describe data capture
Explain about Scanstation & outsourced scanning
Explain that reports are sent out
The next stage is capturing and feedback.
For online surveys participants can submit their responses which will be sent back to the sever and stored in the database.
For paper surveys we use a application called the Scanstation. Questionnaires are printed on normal copy paper from PDF and once completed, put into a scanner ready to be processed. We normal document scanners but recommend and resell Canon scanners as it helps us to support customers if they are using the same models.
Once the forms are in the scanner, the user just presses the big scan button on the Scanstation. The scanner will process the questionnaires and create the images, the Scanstation will the upload the images to the server where the VividForms reader will read the barcodes, read the checkboxes, clip the handwritten comments and then store the results in the database.
The server and Scanstation are always in communication and a status message of each batch will be displayed on screen.
It is possible to set up multiple scanning locations across different buildings, Campuses or countries. There is also the option to outsource scanning to Electric Paper as a service.

14. Instant PDF report Key Points Explain that reports are instant
Explain key features
As soon as you capture one online response or scan one paper questionnaire then EvaSys will instantly create a PDF report for that course/survey.
There are lots of features available in these reports and they can be configured on a questionnaire by questionnaire basis.
It is possible to include a cover letter on the front explaining about the calculations.
The header at the top can be edited, but normally would include the course name, course code, lecturer, number of responses and response rate.
The overall indicators show the aggregated mean and standard deviation for a group of questions. In this example there is an overall module score which shows the mean for all scaled questions.
For every question that you ask EvaSys will create a graphical display of the results.
The default graph for scaled questions is the histogram which includes a bar chart, the percentages on the top, the red line which is the mean and the black line going through it which is the standard deviation. You can also use traffic lights and show the values including the number of responses.

15. Instant PDF report Key Points Explain profile line & comparison
Another feature of the PDF reports is the profile lines which plot the means for the scaled questions and links them together as a line.
This makes it really easy to see any strongly positive or strongly negative responses across all questions, but more importantly, it can be used to show a comparison.
For example you might compare the survey against the last time the course was surveyed, or the department average.
On the slide the red line is the survey and the blue and green lines are the results from previous evaluations.

16. Instant PDF report Key Points Explain comments Distribute reports
Finally at the end of the report you will find the open comments.
For online surveys the comments will be typed. For paper surveys the handwritten comments will be clipped as an image and automatically put into the PDF report. For Hybrid surveys there will be a mix of typed and handwritten comments.
Clipping the images of handwriting means that the processing of paper questionnaires is very fast as there is no delay for data entry.
It is possible to manually key in the comments if you do want them typed though, and it is possible to set a threshold where if less than X participants respond to the survey, then the comments have to be typed before the report is available to ensure that they are not identified by their handwriting.
As already explained, these reports can be ed directly the stakeholders linked the courses in EvaSys. This can either be an automatic process after scanning or the close of an online survey; or it can be a batch process managed by the system administrator.

17. Report Creator Aggregate (Batch Process) Key Points
Describe Report Creator
Explain how to aggregate
After you have distributed your survey reports, EvaSys provides a second level of reporting through the Report Creator. This can be set as part of the Administrator rights or a stand alone user role.
The Report Creator provides the ability to aggregate results based on the course information. For example you might aggregate the responses for all level 2 business courses to get an overall average for the Business School.

18. Report Creator Filter (Batch Process) Key Points Explain how to filter
You can also filter responses based on how the questionnaire has been answered. For example you if you asked a question about gender, you could split male and female repsonses.

19. Report Creator Compare/Email (Batch Process) Key Points
Explain how to compare
Explain batch process
Finally, you can use the profile line features to create comparison reports. For example you might compare all business courses against the Business School average.
This in not a one-at-a-time process, the Report Creator uses batch processing features to make it quick and easy. Comparison reports can also be ed to course stakeholders in a batch process.

20. 3.1 4.1 1 3 4 5 2 - + QM Views Key Points Explain traffic lights
The final reporting level the quality management features.
As discussed, it is possible to set targets on your questions that result in traffic light reporting in the PDF reports.
The way it works is that you have to set two thresholds acrross the scale. In this example 1 is negative, 5 is positive and we have set a threshold at 3.1 and 4.1.
This means that if we capture a mean between 1 and 3.1 we will get a red traffic light, if it is between 3.1 and 4.1 it will be amber and if it is above 4.1 the light will be green.

21. QM Views Key Points Explain dashboards Explain usage
EvaSys will then aggregate these scores and display them in a range of dashboards. Targets can be set per question and you can add more weight to some questions or question groups than others. For example you might weight teaching questions to have a bigger impact on the overall score than questions about facilities.
Here you can see the top 10 highest and lowest indexes.
All the surveys and scores will be presented in a table for which you can filter through and set pre-defined searches.
The green and red circles are a crude indicator of how well the course performed, the more green the more positive the results.
The important figure to note is the Quality Index which is the overall score for the course/survey and is basically a percentage achieved against the targets set.
If there are any concerning scores then you can dig a little deeper by clicking on the details button on the right handside. Here you can see the individual scores and targets for each question. You can even view the handwritten comments in the PDF report if necessary.
These features are generally used to track overall performance of surveys. For example heads of departments might want to drill down through the results and highlight excellence as well as concerns.

22. Summary World leading survey management suite for education
Scalable web-based platform
Paper and online delivery methods
Integrated paper and online data base
Automated reporting and distribution
Quality and performance management
Saves time & reduces costs
Key Points
Summarise
Rubik analogy
In summary, EvaSys is the world leading suite for course evaluation.
It is a feature rich, scalable web application.
It is a hybrid, paper and online system and always has been. It is not a paper system that has bolted on some online functionality like the old OMR scanners.
And, as discussed it includes lots of reporting and automation features.
Not only is EvaSys designed to save time and reduce costs, but it provides institutions with many other benefits such as overall control and visibility of response data.
You may know that we hold regular seminars on the topic of effective course evaluation where we present EvaSys and talk about best practices. Our customers often attend and talk about their experiences with EvaSys. One of our contacts from the University of Hull used a good analogy to summarise a key benefit of implementing EvaSys. They said that before EvaSys there was no structure and monitoring evaluations were a challenge. If you consider that each side of cube is a different department and each square and colour and different course or survey process – it was chaotic!
However, since implementing EvaSys, the process is better organised and the university has better visibility as to what is being surveyed and when.

23. Thank you!