1. Acceptance Testing and Quality Assurance of Radiation Treatment Planning Systems
James A. Purdy, Ph.D.
Professor and Director
Division of Radiation Physics
Department of Radiation Oncology
Washington University School of Medicine
St. Louis, Missouri, USA
Evolving Strategies in Radiation Oncology
Rome, Italy, June 3-5, 2004

2. Acceptance Testing and QA of Treatment Planning Systems
Treatment planning and delivery of radiation therapy are undergoing significant change with continuous advancements in computer hardware computational power and computer graphics.
Image-based 3D planning including IMRT inverse planning is driving a complete shift in the paradigm for the treatment-planning process.
Can be very large difference in functional capabilities and clinical utilization of planning systems.
Acceptance testing and commissioning of planning system is quite complex and may vary from institution to institution.

3. Acceptance Testing and QA of Treatment Planning Systems
Basic Components
computer workstation
input and output devices for graphics and images
Complex parts of a planning system are
dose-calculation algorithm code
programs used to manipulate 3D graphic displays of the patient, beam geometry, and dose
optimization engine

4. Acceptance Testing and QA of Treatment Planning Systems
Testing all components of a treatment-planning process can be a formidable task.
Physicist must ascertain extent and complexity of treatment-planning needs of clinic
Based on this information, physicist must establish elements of acceptance, commissioning, and QA of the 3D RTP system.

5. Acceptance Testing and QA of Treatment Planning Systems
Van Dyk et al.: Commissioning and QA of Treatment Planning Computers. IJROBP 26, , 1993

6. Acceptance Testing and QA of Treatment Planning Systems
AAPM Task Group 40:
Comprehensive QA for Radiation Oncology. Med. Phy. 21, (1994)

7. Acceptance Testing and QA of Treatment Planning Systems
AAPM Task Group 53:
Quality Assurance for Clinical Radiotherapy Treatment Planning. Med. Phy. 25, (1998)

8. Acceptance Testing and QA of Treatment Planning Systems
Guidance document on delivery, treatment planning, and clinical implementation of IMRT: Report of the IMRT subcommittee of the AAPM Radiation Therapy Committee. IMRT CWG. Med. Phys., 30(8): , 2003.

9. Acceptance Testing and QA of Treatment Planning Systems
QA program must consider how RTP system is used as well as how it interacts with the treatment planning process.
Creation of a treatment planning process that incorporates self-consistency and procedural checks is a major component of a QA program for treatment planning.

10. Acceptance Testing and QA of Treatment Planning Systems
Physicist must be afforded adequate time to ascertain extent and complexity of treatment planning needs of radiation oncology clinic.
Based upon this information, physicist must design and implement an appropriate QA program.
For a treatment planning process of a given complexity, QA requirements in a small radiation oncology facility should be no less than those in a large, academic medical center.

11. Acceptance Testing and QA of Treatment Planning Systems
performed to confirm that the RTP system performs according to its specifications.
If there is little rigor in the specifications of the RTP system, then there will be little need to design an acceptance test.

12. Acceptance Testing and QA of Treatment Planning Systems
AT specification divided into 3 categories:
Computer hardware: Includes CPU and all peripheral devices that are part of RTP system, such as display monitor(s), printer, plotter, etc.
Software features and functions: Many software feature specifications will be of the yes/no or exists/does-not exist type, rather than quantitative.
Benchmark tests: Performance on benchmark tests indicates accuracy of dose calculation algorithm under very specific circumstances with specific beam data. Calculation times can also be measured.

13. Acceptance Testing and QA of Treatment Planning Systems
Test procedures document should be written that clearly describes individual procedures in detail.
Order of tests important to minimize total work necessary and to correlate optimally with other tests.
Procedures to be used must be agreed to by both the user and the vendor.
Acceptance testing should be carried out on the system after it has been installed in the clinic but before it is used clinically.

14. Acceptance Testing and QA of Treatment Planning Systems
Significant time may be required to perform detailed benchmark testing of dose calculation algorithm accuracy.
Most tests of the hardware and the software features should be performed by the user.
If some tests are performed by vendor, user may want to repeat some or all of the tests to verify results.
Results from acceptance testing should be carefully documented, along with any variation from defined procedures, and kept as long as RTP system is used in the department.

15. Acceptance Testing and QA of Treatment Planning Systems
Modern RTP process includes many aspects not directly related to dose calculations.
RTP QA program must also handle these important non-dosimetric issues.
Non-dosimetric issues apply to 2-D systems and 3-D planning systems.

16. Acceptance Testing and QA of RTPS: Image Input Tests
Image Geometry:
Document and verify parameters used to determine geometric description of each image (e.g., number of pixels, pixel size, slice thickness)
Geometric Location and Orientation of scan:
Document and verify parameters used to determine geometric location of each image (i.e., left- right and head-foot orientations.

17. Acceptance Testing and QA of RTPS: Image Input Tests
Text Information: Verify that all text information is correctly transferred
Incorrect name or scan sequence identification could cause misuse or misinterpretation of scans.
Imaging Data: Verify accuracy of grayscale values, particularly for conversion of CT number to electron density.
Wrong grayscale data may cause incorrect identification of anatomy or incorrect density corrections.

18. Acceptance Testing and QA of RTPS: Image Input Tests
Image Unwarping (Removing Distortions): Test all features, including the documentation tools which assure that the original and modified images are correctly identified within the system.
Methodologies which modify imaging information may leave incorrect data in place.

19. Acceptance Testing and QA of RTPS: Contour Tests
Manual contour acquisition
Digitization process (hardware & software)
Contouring on 2-D images
Autotracking contours
Bifurcated structures
Contours on projection images (DRRs, BEVs)
Contours on CT scannograms
Extracting contours from surfaces

20. Acceptance Testing and QA of RTPS: Image Use and Display Tests
Verify functionality of window and level setting.
Determine whether displayed window/level values agree with those on scanner/film.
Verify accuracy of the geometric location of the image.
Verify accuracy of the grayscale reconstruction and of any interpolation performed during that reconstruction.
Check consistency between new images and original images.
Verify the capability to remove unwanted imaging information, such as the patient support table.

21. Acceptance Testing and QA of RTPS: Image Use and Display Tests
Verify accuracy of the geometrical location of the slices with respect to the rest of the patient anatomy.
Verify mean, minimum and maximum CT number inside a region of interest (slice and volume) for a range of situations.

22. Acceptance Testing and QA of RTPS: Image Use and Display Tests
Verify point coordinates, distances and angles in each coordinate system for each display type.
Confirm color and other rendering functions.
Verify that each panel of a multiple window is kept current as the planning session proceeds.

23. Acceptance Testing and QA of RTPS: Beam Description/Configuration Tests
Verify that all beam technique functions work, using a standard beam description provided by the vendor.
Verify that library of available machines and beams is correct.
Verify that availability of machine and beam-specific accessories, such as electron cones or wedge is correct.

24. Acceptance Testing and QA of RTPS: Beam Description/Configuration Tests
Verify correct use and display of user- defined beam names and numbers.
Verify correct use and display of angle readouts for gantry, collimator, and table.
Verify correct use and display of linear motion readouts of table, collimator jaws, and MLC.
Check readout names and motion limitations.

25. Acceptance Testing and QA of RTPS: Beam Description/Configuration Tests
Verify correct functionality of tools such as those to move isocenters or set SSDs.
Verify correct wedge characterizations (coding, directions, field size limitations,and availability).
Verify correct use and display of compensators.

26. Acceptance Testing and QA of RTPS: Beam Geometry Display Tests
BEV/DRR Displays
Verify projection of contours/structures defined on axial slices into BEV-type displays. (Compare with the grayscale images for DRR displays. This is most easily done with a simple phantom containing only a few internal structures).
Verify projection of divergent beam and aperture edges.
Check at several different SSDs and projection distances.

27. Acceptance Testing and QA of RTPS: Beam Geometry Display Tests
Beam Display: Verify that
Positions and field sizes are correct.
Wedges are shown and the orientation is correct.
Beam edges and apertures are shown correctly.
Patient and Beam Labels
Verify patient orientation with respect to beam and orientation annotations.
Verify correctness of orientations and annotations for machine position views or icons associated with 2-D or 3-D displays.

28. Acceptance Testing and QA of RTPS: Field Shape Design Tests
Verify that system distinguishes between "island" blocks, in which aperture delineates block shape, and"aperture" or "conformal" blocks, for which the drawn aperture encloses the open irradiated area.
Verify correct specification of transmission or block thickness for full blocks and partial transmission blocks.
Test methods used to fit MLC leaves to field shape.
Verify availability/size of electron applicators.
Check all output showing beam apertures used for beam aperture fabrication (e.g., MLC leaf positions, BEV plots).

29. Acceptance Testing and QA of RTPS: Wedge Tests
Confirm that wedge orientation and angle specifications are consistent throughout RTP system, including hardcopy output. (If possible, they should agree with treatment machine conventions).
Check display of wedges in different 2D planes for different beam directions, coll. rotations, wedge orientations.
Check display of wedges in room view 3D displays for situations described above.
Verify that wedge orientations and field sizes not allowed by treatment machine (energy) are not allowed in RTP system.

30. Acceptance Testing and QA of RTPS: Wedge Tests
Autowedges (wedges inside the head of the machine) : Confirm that the division of a field into fractional open and wedged fields agrees in the RTP system and on the treatment machine.
Dynamic wedge. Verify that the implementation in the RTP system has the same capabilities, limitations, orientations and naming conventions as on the treatment machine.

31. Acceptance Testing and QA of RTPS: Dose Calculation Tests
Verifies that dose calculation program correctly computes doses with respect to algorithm used
Documents limitations of algorithm and its’ implementation
Reviews all input data for all radiation sources for consistency and validity
Provides information required to guide treatment decisions based on documented algorithm limitations

32. Acceptance Testing and QA of RTPS: Dose Calculation Tests
Photon Beam Dose Calculations: Perform dose calculations for a standard photon beam dataset. Tests should include:
various open fields
different SSDs
blocked fields
asymmetric jaw fields
MLC-shaped fields
wedged fields
inhomogeneity test cases
multi-beam plans
other tests

33. Acceptance Testing and QA of RTPS: Dose Calculation Tests
Electron Beam Dose Calculations: Perform a set of dose calculations for a standard electron beam dataset. Tests should include:
open fields
different SSDs
shaped fields
surface irregularities
inhomogeneity test cases
other tests

34. Acceptance Testing and QA of RTPS: Dose Calculation Tests
Brachytherapy Dose Calculations: Perform dose calculations for single sources of each type, as well as several multi-source implant calculations. Tests examples include:
standard implant techniques such as a GYN insertion with tandem and ovoids
two-plane breast implant
prostate seed implant

35. Acceptance Testing and QA of RTPS: Dose Display Tests
Dose Points: Verify that
point is defined at the desired 3-D coordinates.
point is displayed at the correct 3-D position.
dose at point is displayed correctly.
Interactive Point Doses: Verify that
point coordinates correctly correspond to cursor position on display.
Consistency: Verify that
doses in intersecting planes are consistent.
different dose display techniques are consistent.

36. Acceptance Testing and QA of RTPS: MLCs - Dose within the Field Tests
Leakage through leaf (~2%)
Between neighboring leaves (~5%)
At abutting leaf pairs (15% or higher if rounded ends)

37. Acceptance Testing and QA of RTPS: MLCs - Dose within the Field Tests
MLCs must be very accurate for subfields to add together properly

38. Leaf Positioning and Gap Width
Dose delivered with IMRT is very sensitive to errors in the calibration of leaf position
Segment 1
Segment 2
Segment 3
Segment 4

39. Leaf Calibration and Film Test
1) Displacement of both leaves
2) Overtravel of Left leaf
3) Overtravel of Right leaf

40. Acceptance Testing and QA of RTPS: Dose Display Tests
Dose Grids: Verify
Dose is correctly interpolated between grid points for both small and large spacing.
2-D Dose Displays: Verify
Isodose lines (IDLs) are correctly located.
Colorwash display lines up correctly with IDLs and agrees with point dose displays.

41. Acceptance Testing and QA of RTPS: Dose Display Tests
Isodose Surfaces: Verify
Surfaces are displayed correctly.
Surfaces are consistent with isodose lines on planes.

42. Acceptance Testing and QA of RTPS: IMRT Commissioning
All tests that are required to verify a 3D-TPS
TG-53 requirements
Accurate beam data is essential IMRT modeling
Penumbra model is extremely critical
Small field output factors are required
Combination of micro-chambers, diodes and film may be used
Film, TLD, MOSFET, diode and Ion-chamber dosimetry are used for commissioning tests

43. Acceptance Testing and QA of RTPS: IMRT Commissioning
MLC leaf positioning accuracy
Better than a millimeter
MLC leaf end location
Gap width
Beam characteristics for small MUs
Dose/MU constancy
Flatness and symmetry of the beam
Leaf speed accuracy

44. Acceptance Testing and QA of RTPS: IMRT Commissioning - Plane Film Analysis
RTPS applies the planned fluence on a solid water phantom at a known depth
Computes the dose at that depth and generates a dose map file
Expose a film – Convert OD to dose
Compare with dose map generated by RTPS
Plain Field 1
Plain Field 3

45. Acceptance Testing and QA of RTPS: IMRT Commissioning - Total Plan Validation
Transfer the patient plan to a phantom
Treat the phantom and measure the doses
Ion-chambers, Films, MOSFET dosimeters
Compare the doses generated by the TPS

46. Acceptance Testing and QA of RTPS: Dose-Volume Histogram Tests
Structure Identification
Test Boolean combinations of objects (VROI and DVH Normal Tissue - Target), and how voxels which belong to multiple structures are handled.
Normal Structure
GTV (target)
PTV (target)

47. Acceptance Testing and QA of RTPS: Dose-Volume Histogram Tests
Histogram Bins and Limits
Verify appropriate histogram bins and limits are used.
DVH Calculation
Test DVH calculation algorithm with known dose distributions.
DVH Types
Verify that differential, and cumulative type of DVH are calculated/displayed correctly.
DVH Plotting/Output
Test DVH plotting/output using known dose distributions.

48. Acceptance Testing and QA of RTPS: Dose-Volume Histogram Tests
Dose Display, Dose-Volume Histograms
User-created volumes/dose distributions may also be used for additional tests.
Int Struct
TV1
TV2
TV3
Unspec. Tissue

49. Acceptance Testing and QA of RTPS: Multiple Beam Dose Distributions Tests
Multiple beam dose distribution summation (and subtraction - deletion)
Arc beam simulation and verification (for algorithms which use static field superposition)
Spot check isodose plots and display using point dose samples and comparing to overlying curves

50. Acceptance Testing and QA of RTPS: Hardcopy Output Tests
Print out all hardcopy documentation for a given series of plans, and confirm that all textual and graphical information is output correctly.

51. Acceptance Testing and QA of RTPS: Connectivity Tests
Linac
CT Scanner
Virtual Simulation
Workstation
Treatment Planning
Workstation
MLC Controller
Record & Verify
Film
Printer
Network
CT/MR Image
CT Image
RT Structure Set
MR Scanner
RT Plan
RT Plan
RT Image
It is imperative that the RTPS vendors demonstrate the connectivity as a part of the system acceptance testing

52. Acceptance Testing and QA of RTPS: Summary and Conclusions
Physicist must establish elements of acceptance, commissioning, and QA of the RTP system.
Software validation testing should be independent of the manufacturer’s tests
Dose calculation verification testing should be independent of the manufacturer’s tests
Alignment and immobilization systems must be in place to enable accurate, reproducible implementation of 3DCRT/IMRT planning.

53. Acceptance Testing and QA of RTPS: Summary
Plan implementation and treatment verification systems and procedures must be in place to augment RTP system QA program
Physicians, physicists, dosimetrists, and therapists must work as a team to ensure a safe, high quality treatment planning and delivery system

54. Thank you for your attention.