Table 1 Sample ML-aided applications that are limited by the individual non-technological challenges and a summary of the potential solutions to those challenges
From: Non-technological barriers: the last frontier towards AI-powered intelligent optical networks
Type of non-technological challenge | Example ML-aided applications affected | Summary of proposed solutions |
|---|---|---|
Legacy issues | • Quality of transmission estimation15,16,17,45 • Network failures management40,41,42 | • Conveying long-term cost-saving potentials of ML-enabled intelligent solutions to the network operators • Establishing communication channels between solution developers and network operators to convey what ML-aided tools are able to achieve realistically and how they can help gain advantage over other competitors |
Cost restraints | • Optical performance monitoring25,26 • Quality of transmission estimation | • Curtailing data costs by exploiting data readily available in coherent receivers as a by-product of DSP modules • Exploiting cheap synthetic data from simulation tools • Reducing data dependency by applying less data-hungry ML algorithms |
Expert workforce limitations | • End-to-end communication system optimization • Network failures management | • Developing interdisciplinary education programs supported by industry and academia to prepare expert workforce adept in both optical communications and ML • Introducing collaborative ML skills development programs backed by industry stakeholders to produce required workforce of field engineers and technicians |
Data accessibility and privacy protection problems | • Network security management • Network failures management | • Identifying clear benefits of sharing real networks data to incentivize the network operators to share their data • Defining protocols and clear ToU for proprietary network data to encourage trustworthy data sharing • Protecting data privacy in collaborative network applications through federated learning framework |
Interpretability, transparency and accountability issues | • Photonic components design • Fiber nonlinearity compensation8,9,10 • Network failures management | • Using comparatively simple but comprehensible ML methods for less complicated network applications • Applying interpretability frameworks to explain the behavior of intricate and non-transparent DL methods developed • Enabling accountability by disclosing all information related to data generation processes, details of the data sets used, and ML models’ development procedures |
Lack of standardization and regulatory frameworks | • Quality of transmission estimation • Short-reach data centre networking76,77 • Photonic components design | • Defining standards for data generation processes, data sets specifications for various applications, ML models’ performance metrics and testing conditions, etc., through cooperation of industry and standardization organizations • Formulating regulatory policies for data market (e.g., obligating datasheets for all data sets) and ML models (e.g., warranting report cards for developed ML models) through joint efforts of industry and governmental bodies |
Human factors and cognitive biases | • Network failures management • End-to-end communication system optimization (e.g., which specific transmission factors are not modeled in the end-to-end optimization process is decided by the humans leading to certain performance penalties) | • Diversifying workforce responsible for data generation processes and aggregating inputs from multiple data sources to minimize the risk of data biasness • Providing datasheets that confirm checking and absence of any data biases and also give links of original data sets so that developed tools could be independently verified • Detecting hidden biases in data using statistical tests and eliminating them through various debiasing techniques |
