Certificate handling and secure key storage ONAP SECCOM F2F, Kista, June 11-14, 2019 Ericsson

Agenda Certificate handling (~40 min) Secure key storage (~50 min) Certificate handling for external interfaces Hampus, Ericsson Secure key storage (~50 min) AAF current solution Jonathan, AT&T ISTIO CA private key storage Pramod, Intel

Background certificate handling For avoiding confusion this material will be focused in setting up external interfaces. Several components uses third party software for setting up mTLS with xNF. ONAP B A C xNF = internal (K8s) = external (xNF)

Background from Dublin Currently not a centralized solution as there are components not using AAF for external mTLS AAF CA is made available and AAF currently generates a certificate which can be installed into, e.g DCAE. – link This functionality is primarly intended to be used for internal ONAP communication. No seperate files or certificates are provided for external mTLS (including trusted cert). Each component has a slighly different solution including file format (PEM/PKCS12/JKS) With further background from Dublin, and we would love to get more experiences from developers,.. there have not been a aligned solution as components are not using AAF for external interfaces. Which can be seen in the documentation for Dublin components regarding their keys and certificate handling. This does not mean that components are not interacting with AAF at all. For instance is the AAF CA available and currently generates certificates which can be installed, in say DCAE. So in principle the DCAE collectors (VES/HV-VES/DFC) could be configured to point to the files already today generated by AAF, but they are primarily intended for ONAP internal communication. As, there are no separate files or certificates (included trusted certs) provided for external TLS, such as trusted cert. Mainly due to the lack of an external certificate handling protocol for ONAP. As a result of this non centralized solution, each component has today a slightly different solution, including file format (PEM/PKCS12/JKS), This typically indicates that different companies have developed the TLS functionality. As the files must be prepared by the service provider outside of ONAP, they could either contain manually signed certificates or certs signed by preferred external CA. With regards to O&M, I do believe that for certificate handling, I at least, would like to encourage the community to strive for a centralized solution, also for external interfaces using standardized protocols. DCAE VES collector Documentation DCAE HV-VES collector DCAE Data File collector Documentation SDNC/OpenDaylight

Background from Dublin targets Project Description AAF AAF is able to be initialized as RA to the Service Provider (SP) CA. AAF provides an API that allows SO (or some ONAP component) to trigger the IAK/RV scenario execution and returns the IAK, RV and CA URL. AAF is able to execute the scenario to obtain an Initial Authentication Key (IAK) and Reference Value (RV) from a SP CA on behalf of a VNF. AAF is able to act as a CA for lab/test environments where there is no SP CA and to be able to provide X.509v3 certificates to VNFs using CMPv2 protocol as specified by RFC 4210. Security communications between xNFs and ONAP (PA3)

Certificate handling for External interfaces Standalone PKI mgmt CA Hierarchy CA Hierarchy Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 PNF ONAP AAF RA mTLS … DCAE Trusted Cert. Trusted Cert. SDN-C

Certificate handling for External interfaces Standalone PKI mgmt CA Hierarchy CA Hierarchy Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 CMPv2 PNF ONAP AAF RA mTLS … DCAE Trusted Cert. Trusted Cert. SDN-C

CMPv2 recommendations according to 3GPP RA/CA base station The base station obtains the operator-signed certificate on its own public key from RA/CA using CMPv2. CMPv2 Vendor-signed certificate of base station public key is pre-installed. Vendor root certificate is pre-installed. SEG Operator root certificate is pre-installed. Enrolled base station certificate is used in IKE/IPsec. IPsec 3GGP cleary states that CMPv2 is the recommended protocol to be used by the radio base station today. This is therefore an already established protocol for PKI management that vendor CA and operator CA’s today already supports, if they follow 3GPP. 3GPP: TS 33.310

Certificate handling for External interfaces Standalone PKI mgmt CA Hierarchy CA Hierarchy Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 CMPv2 ONAP CMPv2 - Use Cases ONAP internal components to receive certificates from external CA that already today uses CMPv2. xNF might require certificate via AAF and external CA according to 3GPP. xNF ONAP With the already established CMPv2, this should be used in ONAP for easier external CA integration and 3GPP compliance. AAF RA mTLS … DCAE Trusted Cert. Trusted Cert. SDN-C

CMPv2 recommendations according to 3GPP CMPv2 protocol should be used as specified in RFC 4210 and RFC 4211 As RA/CA, an operator may deploy: Alternative 1: Stand-alone CA implementing a CMPv2 server Alternative 2: CA having delegated certain tasks to an RA, which in this case operating the CMPv2 server . 3GPP: TS 33.310

CMPv2 solution alternative 1 Standalone PKI mgmt CA Hierarchy CA Hierarchy CMPv2 server Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 Pros Easier integration for ONAP components to have one centralized CMPv2 client. Cons Vendors can not use assumed CMPv2 client in the xNF directly via a RA in ONAP CMPv2 client xNF CMPv2 client ONAP In the first alternative, a CMPv2 server will be located in the external CA. The CMPv2 client will then be centralized located in the AAF. Please note as for this alternative there will be no true RA in ONAP. At least I have had the assumption this might be what ONAP is to target, but it has been a bit missleading as we have still called it an RA so I was hoping we could clear this out. AAF RA mTLS … DCAE Trusted Cert. Trusted Cert. SDN-C

CMPv2 solution alternative 2 Standalone PKI mgmt CA Hierarchy CA Hierarchy Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 CMPv2 Pros Vendors can utilize assumed CMPv2 client in the xNF directly via a RA in ONAP Cons CMPv2 client in ONAP components requires implementation support CMPv2 server xNF CMPv2 client ONAP For the second alternative we then instead has the CMPv2 server located centralized inside the AAF, acting as a RA. In this scenario, it would be logical for the components to act as CMPv2 clients. Thereby, aslo be the entitity to generate the Certicate Signing Request and be responsible for sending up requests to the RA and CA. Currently the CSR is only created by the AAF for internal communication. Note: With an RA you are actually moving down functionality from the CA. What kind of functionality that is moved down might differ depending on systems. Note: Provisioning of IAK and RV are required to all ONAP compents as CSR should securerly being able to be sent to an external CA. CMPv2 client AAF RA mTLS CSR … CSR CMPv2 client DCAE CSR Trusted Cert. Trusted Cert. SDN-C CSR CMPv2 client

Background from Dublin targets Project Description AAF AAF is able to be initialized as RA to the Service Provider (SP) CA. AAF provides an API that allows SO (or some ONAP component) to trigger the IAK/RV scenario execution and returns the IAK, RV and CA URL. AAF is able to execute the scenario to obtain an Initial Authentication Key (IAK) and Reference Value (RV) from a SP CA on behalf of a VNF. AAF is able to act as a CA for lab/test environments where there is no SP CA and to be able to provide X.509v3 certificates to VNFs using CMPv2 protocol as specified by RFC 4210. Conflict Valid Valid As of Dublin targets. Regardless of CMPv2 solution AAF will still be required to support previously detailed IAK/RV functionality. However, depending on the chosen CMPv2 architecture there might not be a true RA in place,. Thus, the other two targets would not apply. Conflict Security communications between xNFs and ONAP (PA3)

Certificate handling for External interfaces Standalone PKI mgmt CA Hierarchy CA Hierarchy Suggested requirements on PKI for ONAP Frankfurt ONAP components shall use a centralized certificate management solution. AAF must support CMPv2 AAF must support installation of certificates required for external mutual TLS using CMPv2 RA CMPv2 CMPv2 xNF ONAP To the third suggested requirement, ”AAF must support installation of certificates required for external mutual TLS using CMPv2”. Depending on how the community decides how to utilize CMPv2 this requirement might be modified. Say if the components themself where to have a cmpv2 client, then AAF would have a little to do with this installation as it would more act as RA. Regardless how CMPv2 is used it is important to highlight the need of keeping the requirement on AAF to be able to generate an IAK/RV and distrubute it to the VNF. AAF RA mTLS … DCAE Trusted Cert. Trusted Cert. SDN-C

Additional items for external certificate handling Some ONAP compents acts as mTLS servers, others as mTLS clients. Suggestion is to include an investigation in the ONAP communication matrix work item– SECCOM-55 Such documentation could make xNF and certificate handling easier. Could also be included in a future ONAP Security architecture document. ONAP secure key storage integration for 3PP handling external interfaces. ONAP B A C This might also influence the solution for how CMPv2 are to be use. xNF = internal (K8s) = external (xNF)

Secure key storage A centralized solution for secure key storage, KMS, that ONAP projects easily can integrate could be interesting for future release AAF is the logical place to handle this in a centralized manner as it will be the place of the “RA” and other security functionality. I will not go into further details for why this is important to have as I assume we can all agree on this.

Secure key storage It seems we are having two seperate tracks that are investigating this. In the ONAP wiki we do have the Certificate and Secret Management Service (CSM), which has secure key storage in scope and it seems to be active. However, the SMS that is mentioned in the AAF release document states that they do not have secure key storage in scope. Instead of introducing a third track, let us sync community views. AAF current solution ISTIO CA private key storage   I could understand why there are people confused for where secure storage is made, as I also managed to confuse myself I guess this is outdated info and that latest readthedocs for SMS is more relevant. However, it seems there are still activity regarding the initial scope for CSM. Also when asking CMS/SMS/Intel people about the SMS they referred to a investigation currently taking place for secure storage of private keys. This was for CA private key protection with what I believe was using SoftHSMv2, however then for ISTIO CA. Srini referred to that Manjunath is driving such activity and asked him to present here in the F2F.

References Security Dublin Priorities (PA4) xNF Communication NETCONF TLS (SECCOM 2019-02-20) 3GPP TS 33.310 (v16.1.0) Security communications between xNFs and ONAP (PA3)

Thank You!

Problem statement When a VNF is instantiated, it needs a way to obtain a certificate from the Service Provider’s (SP) CA that allows it to communicate with DCAE (via HTTP/TLS). PNFs generally come with vendor certificates issued by the vendor’s factory CA that use the PNF serial number or MAC address as the identity. This allows a SP to pre-provision the vendor’s factory root certificate into its CA and even pre-provision the PNF identity to authorize the PNF to request a certificate. This pre-provisioning is not possible for a VNF. A VNF has no vendor certificate and no inherent identity. Two options are supported by 3GPP for VNFs using CMPv2 as described in RFC 4210. Option 1: PKCS#12 bundle is installed on the VNF at instantiation time. A proxy entity performs Certificate Enrollment with the CA on behalf of the VNF to request a PKCS#12 bundle from the CA and this is installed in the VNF during or after instantiation. Option 2: VNF performs certificate enrollment with a one time use Initial Authentication Key (IAK) A proxy entity requests an IAK and Reference Value (RV) from the CA on behalf of the VNF. The IAK and RV are provisioned on the VNF during or after instantiation. After instantiation, VNF performs certificate enrollment with the CA via CMPv2 using the IAK to protect the Certificate Signing Request (CSR). The RV identifies the IAK to the CA. Option 2 is the preferred solution. AAF acts as the RA for the VNF in commercial deployments. AAF provides an API to trigger AAF to obtain an IAK and RV on behalf of the VNF. AAF requests an IAK and RV on behalf of the VNF during the instantiation workflow and passes these to the VNF during instantiation. AAF acts as the CA for VNF certificate enrollment in an ONAP test/lab environment.

Stage B : Service Instantiation Part 1 Precondition: At ONAP deployment time, AAF CertMan is configured to be a Registration Authority (RA) for the operator’s CA. Service Provider (SP) decides to instantiate a new service instance in ONAP. VID User or BSS/OSS system triggers Service Instantiation which includes a VNF. SO triggers OOF for Homing assignments. This assigns the VNF to a cloud location. SO creates AAI entries for Service and VNF. SO calls Openstack to assign cloud resources. Note: Alternately, SO may call M-Cloud which calls Openstack (TBD). SO updates AAI entry with cloud resources. SO triggers SDN-C to create the network connections that Service needs. SDN-C creates network and assigns UUID. SDN-C updates AAI entry. SDN-C notifies SO that connections are created. SO triggers AAF to gets an IAK/RV for the VNF, the CA root certificate, and the CA URL (new AAF API) AAF gets an IAK/RV for the VNF, the CA root cert, and the CA URL. BSS / OSS / VID SO OOF AAI SDN-C AAF OpenStack B1 New Service B2 B1 Service Instantiation B2 B3 Homing Assignments B3 B4 Create AAI Entry for Service and VNFs B4 B5 Assign cloud resources B5 Update AAI with cloud resources B6 B6 B7 B7 Trigger create NW connections B8 B8 Create network, assign UUID, update AAI B9 B9 NW connections complete B10 B10 Request VNF IAK/RV, CA root cert, CA URL Get IAK/RV for VNF, CA root cert, CA URL B11 B11

Stage C: VNF Security Credentials SO AAI AAF CA Stage C: VNF Security Credentials C1a Submit initialization request (ir) to CA Create request for IAK/RV C1b IAK, RV Verify request came from AAF Create IAK and RV C1c C1d IAK, RV, CA root cert, and CA URL C1e Request VNF IAK/RV B8 VNF IAK/RV Obtained from CA SO triggers AAF to request IAK/RV from CA. (New API) AAF creates a request for IAK/RV. AAF submits request to CA. CA verifies request is from the RA and creates response containing IAK and RV. CA sends response containing IAK, RV. AAF returns IAK, RV, CA root cert and CA URL to SO B8 C1a C1b C1c C1d C1e This flow is based on Exhibit 92.3 of Information Security Handbook

Stage E : VNF Certificate Enrollment Certificate Enrollment with IAK/RV Certificate Enrollment can occur after VNF instantiation if the necessary information is passed during instantiation (CA URL, IAK, RV, CA root certificate, VNF FQDN). Else it can be performed during Activation if the necessary information is configured during Service Configuration. Note that the VNF must automatically perform certificate enrollment. VNF generates a key pair and Certificate Signing Request (CSR), and protects CSR with the IAK. VNF submits IAK protected CSR to CA using the CMPv2 protocol. CA verifies CSR using the IAK and RV and creates response containing a certificate bundle. RV is used by CA to identify the IAK. CA sends certification response containing the signed certificate. VNF sends certificate confirmation. If CA does not receive the certificate confirmation within the prescribed time period, the certificate is revoked. CA sends confirmation ack. VNF AAF CA Generate key pair Create CSR with VNF FQDN as CN and SAN Sign CSR with private key Protect CSR with IAK E1 E2 Request IAK protected signed CSR Initial registration/certification (ir) E1 Verify IAK/RV Verify possession of private key Sign certificate E3 E2 E3 Certification response (ip), Signed certificate E4 E4 E5 Cert Confirmation E5 E6 Confirmation ack E6