1. 5MS Systems Working Group
SWG#3 - Tuesday, 30 October 2018
AEMO Offices:
Level 9, 99 Gawler Place, Adelaide
Level 10, 10 Eagle Street, Brisbane
Level 22, 530 Collins Street, Melbourne
Level 2, 20 Bond Street, Sydney
DIAL IN: ACCESS CODE:

2. Agenda NO Time AGENDA ITEM Responsible Preliminary Matters
2:00pm - 2:05pm
Welcome, introduction and apologies
Hamish McNeish (AEMO)
Matters for Noting
2:05pm – 2:15pm
Actions from last meeting
2:15pm – 2:20pm
System workstream status
Matters for Discussion
2:20pm – 2:30pm
Metering
2:30pm - 2:50pm
Dispatch and Pre-dispatch
2:50pm – 3:20pm
Bidding Format
Tom Butterworth (AEMO)
3:20pm - 3:30pm
Data Model Changes
3:30pm – 3:45pm
Transition
Pierre Fromager (AEMO)
Other business
3:45pm – 3:50pm
General questions
3:50pm – 3:55pm
Forward meeting plan

3. Items for Noting
Hamish McNeish

4. Current Actions
Meeting notes with actions have been ed and published
Any feedback?
Item
Topic
Action required
Responsible By
1.4
Sandbox options
AEMO to evaluate the option of having an external test system that more closely resembles preprod, as opposed to a cutdown Sandbox.
AEMO
24/9/2018
1.5
Sandbox dashboard
AEMO to evaluate having a mechanism such as a dashboard to provide the status of the Sandbox environment. Allow viewing current defects etc…
1.6
Participant bandwidth implications
AEMO to provide assessment on bandwidth/networking impacts
1.8
Retail 1MB file limit
AEMO to propose more information and a recommended limit
2.1
Dispatch delays
AEMO to provide feedback
17/10/2018
2.2
HLIA
AEMO to provide HLIA document
2.3
Focus Group Feedback
Participants to provide feedback on priority areas
SWG Members
2.4
Profiling process explanation
AEMO to explain the profiling process
2.5
Participant feedback on proposed 10 MB limit
21/11/2018

5. Actions to Close
1.4 – AEMO will provide a sandbox environment that will provide end-2-end processes
1.5 – AEMO will provide a webpage indicating the Sandbox status
1.8 – AEMO provided more information and a 10 MB limit at the Metering joint focus group
1.9 – Message Archiving - Participants are open to the concept. This would need to be clarified in AEMO’s design work
2.1 – Dispatch Delay – AEMO discussed this in detail in the Dispatch joint focus group

6. Actions to Close (cont.)
2.3 – Focus group priority – no feedback: remains as is
2.4 – Profiling sample – this will be covered in the metering joint focus group on 12 Nov
2.5 – Feedback on retail 10 MB limit
No concerns identified
AEMO will carry out initial testing during development
A recommendation was received to also test this in industry testing – AEMO supports this.

7. System Workstream - Metering
Planned dates are subject to PWG consultation progress
Early Release depends on PWG outcomes
Changes in red due to Transition joint focus group on 14th Sep
* = Joint focus groups with PWG

8. System Workstream - Dispatch
Planned dates are subject to PWG consultation progress
Early Release depends on PWG outcomes
Changes in red due to planned Transition joint focus group
* = Joint focus groups with PWG

9. System Workstream – Settlement and Operations
Planned dates are subject to PWG consultation progress

10. Matters for Discussion
Hamish McNeish

11. Metering
Hamish McNeish

12. Current Status
First Stage Package 1 (P1) Consultation published on 31 Oct
Focused on 5MS changes and ‘non-mandatory items’, Global Settlements Rule changes will be addressed in Package 2 scheduled for early 2019
Procedures included in P1 consultation included:
Metrology Procedures: Part A
Metrology Procedures: Part B
Retail Electricity Market Glossary and Framework
Meter Data File Format Specification NEM12 & NEM13
Meter Data Provision Procedure
Joint Metering/System focus group on 12 Nov 2018, items being discussed include:
AEMO program transition schedule
Profiling methodologies for 15/30-minute interval metering and controlled load sample meters
MDFF vs MDMF
Agreed interval meter data will be in NEM12/MDFF format
Basic meter data delivery format to be agreed
Register level meter data and data streams
Removal of infrequently used RM reports

13. Dispatch and Pre-dispatch
Hamish McNeish

14. Dispatch and Pre-dispatch timings (avg.)
5-minute
Pre-dispatch
30-minute
Period (cycle)
5 min
30 min
Horizon
60 min
16 to 40 hours 3
Resolution
Process start
t-67 sec
t+0 sec
SCADA Snapshot
t-3 sec
~t+10 sec
Process complete1
t+8 sec
t+36 sec
t+140 sec
Solution published
t+17 sec
t+48 sec
t+155 sec
Data loaded to participants’ database 2
t+22 sec
t+77 sec
t+174 sec
Notes
The process completion and publication times are average times based on normal market conditions. Under conditions such as intervention, where multiple solutions are required, these times will be longer.
These times are sourced from AEMO’s performance monitoring systems. Actual times at participants’ sites will depend on the local infrastructure and configuration.
In the case of the 30-minute Pre-dispatch, there is a larger variation of times around the average due to the variable length of the Pre-dispatch schedule.
Example footer text
25/12/2018

15. 5MS Dispatch Changes Changes
Each bid interval (1 – 288) now relates directly to a dispatch interval
Only the inputs and outputs for bids will change
288 intervals in a bid
“0” fixed load will be treated as 0 not as null
Associated bid data in the Data Model will change, tables may change
Solution files may have some minor changes
Not changing
The existing dispatch process does not change
NEMDE (the solver) formulation does not change

16. Pre-dispatch
Changes
30-minute pre-dispatch will use the last bid in a 30-minute period
e.g. the bid for intervals 6, 12, 18, 24, …, 288
5-minute pre-dispatch will use the bid for interval (1 – 288) that relates to the dispatch interval
Reports will remain unchanged, subject to the options below
Shape of late rebid period is changing to 30 mn before start of interval (from 15) but this moves along every 5 minutes.
Options (Discussed at the joint Dispatch/Systems Focus Group 22 Oct 2018)
Extending 5 minute pre-dispatch (P5) to 6 and/or 24 hours
Including price sensitivities for P5

17. Bidding Format
Hamish McNeish

18. Submission of FTP, Web and API bids
Bid submission will move from TXT based to JSON format
5-mn bids can only be submitted in JSON
30-mn bids will only be supported in TXT format
The Web UI will provide screens to submit either (during the transition period)
Current
Participants currently submit TXT files through FTP (and the Web UI).
Some participants also use the Web UI, submitting data directly to the database.
Proposed
AEMO has identified an opportunity to improve this process by using a more flexible, accurate, text-based format for the submissions, called JSON (JavaScript Online Notation). These would be submitted via
FTP (same process as today),
Web UI (enhanced interface)
APIs or
by submission of a JSON file through the Web portal.
AEMO is also considering accepting CSV uploads via the Web portal (only, not FTP), to support low frequency bidding where systems such as MS Excel can be used to create bidding files.

19. Proposed JSON format
Refer to the separate JSON format document

20. Data Model Changes
Hamish McNeish

21. Bidding Tables
The existing bid table structures can support 288 intervals
Changes are required to support adding APIs and to support transition
The transition from old Data Model reports (30-min) to new (5-min) reports will apply
Table
BIDOFFERFILETRK
Usage
Audit trail of participants’ bid submissions (FTP and WEB), including invalid bids.
Content
Private data
5MS changes
API submissions will be included
A transaction ID field may be added for API submissions
Table
BIDDAYOFFER
Usage
All participants bids for a trading day. Parent table to BIDPEROFFER.
All previous-day bids are also populated to this table.
Content
Private and public data
5MS changes
Add a non-mandatory field to indicate the bid resolution, 5 minute vs 30 minute

22. Bidding Tables (cont.) Table
BIDDPEROFFER
Usage
All participants’ interval-level bid data for a trading day. Child table to BIDDAYOFFER.
All previous-day bids also populated to this table.
Content
Private and public data
5MS changes
See options below
Option 1: Use existing BIDPEROFFER table
New 5 minute reports will be introduced, to transition from 30 minute current reports
A mix of 30 minute and 5 minute data end up in the table
Determining resolution would be done from BIDDAYOFFER
Option 2: Create a new BIDPEROFFER table – e.g. BIDPEROFFER5
Only 5 minute bid data would go into this table
Determining resolution (5-min vs 30-min) would be done from BIDDAYOFFER
The BIDPEROFFER table will become historic from 1 July 2021

23. Bidding Tables (cont.)
The previous-day bid data for the bids used in dispatch are populated into BIDDAYOFFER_D and BIDPEROFFER_D
The same changes will apply to these changes as BIDDAYOFFER and BIDPEROFFER
Table
BIDDAYOFFER_D
Usage
The bids actually used in each dispatch interval for a trading day. The parent table to BIDPEROFFER_D.
Content
Public data
5MS changes
Add a non-mandatory field to indicate the bid resolution, 5 minute vs 30 minute
Table
BIDDPEROFFER_D
Usage
The interval-level bid data used in each dispatch interval for a trading day. Child table to BIDDAYOFFER_D
Content
Public data
5MS changes
Will be the same as the BIDPEROFFER decision

24. Settlement Tables
AEMO proposes to retain the settlements table structures as much as possible
We expect additional information will eventually be published such as UFE for Global Settlement
Some participants have requested 30-minute reconciliation reports be maintained, so they can continue with 30-minute based processes. This would mean:
New 5-minute reconciliation reports would be available to subscribe to
Existing 30-minute reconciliation reports would be updated to aggregate 5 minute level data
Thoughts/comments?

25. Bidding Transition
Hamish McNeish

26. Allow 5-mn bids before commencement date
AEMO’s position:
Accept 5-mn bids from 1 April 2021, 3 months before commencement.
Use these directly in the NEMDE i.e. don’t transform these into 30-mn bids.
For 5-mn bids, prices will continue to apply for the whole trading day. Capacity/availability quantities will be provided for each trading interval (288 intervals).
From 1 April 2021:
existing 30-mn bids for trading days from 1/4/2021 will be converted to 5-mn bids by AEMO (during the release)
any received 30-mn bids will be converted to 5-mn bids by AEMO’s systems
dispatch, pre-dispatch, PASA, etc. will run off 5-mn data.
5-mn bids will be provided in JSON format; 30-mn bids remain in TXT format.
The web portal will support both 30-mn and 5-mn bids during transition.
From commencement, 30-mn bids will be rejected as invalid.

27. Acceptance of 30-mn and 5-mn bids
The following table shows the acceptance/rejection of 30-mn and 5-mn bids.
The Transition Period starts from the release date of 1 April 2021
Commencement is 1 July 2021
Submitted in
Applying to trading days
Prior to 1/4/21
1/4/21 – 30/6/21
From 1/7/21
Pre-transition
Accept 30 mn bids
Transition
N/A: Cannot submit past bids
Convert any existing 30 mn bids to 5 mn.
Accept 30 mn bids – convert them to 5 mn.
Accept 5 mn bids.
From commencement
Reject 30 mn bids.

28. General Questions
Hamish McNeish

29. Forward Meeting Plan
Hamish McNeish

30. Next Sessions SWG Focus Groups
12-Nov-18 – Metering/Systems Focus Group (joint)
Vendor Briefing Session
3-Dec-18
SWG’s
21-Nov-18
17-Dec-18
5MS Meetings and Forum Dates: Minute-Settlement