Introduction

Although the potential benefits of using 3D printing (3DP) are well-reported, challenges with process complexity, reproducibility, and throughput are experienced by most technologies during their development phase. For most 3DP technologies, it is through a decades-long, iterative effort that the understanding of the complex processing dynamics and quality control can be achieved1,2,3,4,5,6. The major focus of this research is instead to leverage the recent developments in manufacturing digitalization and machine learning (ML) to address such challenges through data-driven solutions with an early-stage 3DP technology, melt electrowriting (MEW)7,8,9,10,11.

This particular 3DP technology has the ability to accurately fabricate porous structures using small diameter fibres ranging from 0.3 to 50 µm11,12. MEW is mostly applied in the wider field of bioengineering, more specifically to design and fabricate scaffolds in tissue engineering, cancer research, biofabrication and biomaterial science13. As a multi-parametric process, no quantified universal correlation between input and output parameters in MEW has been established to date. Several studies investigated the effect of input parameters on the output characteristics of the process14,15,16,17. However, many were limited by the lack of in-situ monitoring and control as well as the need for time-consuming, labour-intensive experimentation. As a result, the current MEW printing procedures still involve trial and error methods to find the best process parameters.

Due to the multiparametric and complex, nonlinear relationships between process parameters, the operation of MEW devices requires expert users, which slows the widespread adoption of the technology in academic and industrial laboratories. Therefore, despite its great potential, MEW in its current state is not yet an industry-ready 3DP method. The most consequential input process parameters are flow rate to the nozzle, processing temperature, applied voltage, collector distance, and collection speed14,15,18. Additionally, there are several less critical time-dependent parameters such as material degradation/crosslinking rate19,20, ambient temperature21,22, and ambient humidity22. Conversely, MEW has excellent visual access to important processing information, due to the gap that separates the nozzle and the collector (i.e., the collector distance)23. It is therefore possible to both automatically control the input parameters while measuring the output characteristics of the process, using computer vision and machine learning.

Acknowledging the successful application of digitalization and real-time quality control in other industry-ready 3DP technologies, research in MEW has started to shift the focus to process automation and in-situ process control23. With a camera vision-based system, process analysis is sped up using real-time process monitoring and image processing algorithms15,24. Identifying the Taylor cone area as an output parameter representing fibre diameter, real-time process monitoring could analyse process stability and ultimately define the relationship between the jet angle, Taylor cone area, and fibre diameter23. The motivation behind these recent advancements is to attain a better understanding of the MEW process for enhancing the technology readiness level of the innovative 3D printing technology based on improving reproducibility and automated throughput. The present study expands on previous research by introducing a semi-automated methodology to obtain a non-linear model of the MEW system using artificial neural networks (ANNs), which is subsequently used to develop closed-loop control of the process.

Decisively, this work addresses the main technological challenges that MEW is currently facing:

(1) slow and labour-intensive experimentation, (2) low throughput of the printing process, (3) poor reproducibility of the printed structures, and (4) highly complex operation of the MEW system. Slow and operator-dependent experimentation is addressed through the automated process monitoring and control platform. The system allows for real-time parameter monitoring, control, and analysis via advanced hardware and computer vision algorithms. The ability to rapidly perform a series of experiments allowed us to collect high volumes of input-output data by testing multiple configurations of printing parameters. That data was utilized to construct a model of the MEW system using ANNs. The resulting model was used to provide feedforward control of the input parameters given the desired output parameters, which reduces the complexity of operating the printer. The ANN model was also used as the foundation of a closed-loop controller that ensures reproducible results by minimizing the output error in real-time. Overall, the presented solution and methodology integrate the latest MEW hardware and ML techniques to develop, for the first time, a data-driven closed-loop control system to enhance throughput and reproducibility.

The MEW jet is a catenary shaped fluid that is stabilized by the applied voltage, preventing the fluid from breaking into droplets due to Plateau-Raleigh instabilities. The jet angle is defined as being between the vertical line through the centre of the nozzle and the line that intersects the jet contact point with the collector. The jet angle dictates the accuracy of the material deposition, especially when the jet is deposited with changing directions24. The Taylor cone area is an output characteristic that reveals the information on printing stability but can also be correlated with fibre diameter—the metric defining the size of the printed fibres23. Figure 1A shows a simplified diagram of the MEW process as well as the input and output process parameters of the system. Figure 1B shows the assembled MEW device highlighting the most critical hardware components, while Supplementary Fig. 1 provides a magnified view.

Fig. 1: Schematic of the MEW jet.
figure 1

A The molten polymer in a heated syringe is pushed through the nozzle via pneumatic pressure, forming a droplet called a Taylor cone. The electrostatic charges result in a fine jet to be ejected from the Taylor cone toward the collector. This thin electrified jet is direct written on to a surface (the collector) which solidify into microfibres. Red text indicates input process parameters and blue text indicates key output process parameters. B Photograph of the automated MEW 3D printer with the most important hardware components annotated, including X, Y, Z-axis stages, print head, collector, collimated backlight and telecentric lens/camera.

Full size image

Results

MEW hardware design and characterization

Currently, MEW hardware offers little process monitoring, control and automation, which are necessary to mature the technology towards a higher technology readiness level and ultimately industrial application. Performing experiments with minimal user intervention is essential to rapidly collect large volumes of data that can be used to create an ML model of the system dynamics and ultimately model-based control. MEW provides excellent visual access between the nozzle and the collector, which provides important information on the material deposition process. To exploit this information, a telecentric vision system was designed and implemented to obtain a detailed, high-resolution video feed of the fabrication process. A telecentric lens (Edmund Optics, USA) and monochrome camera (Basler, Germany) are coupled with collimated backlight illumination (Metaphase Technologies, USA) to obtain high-quality images of the polymer deposition process (see Supplementary Fig. 2 for more details regarding the telecentric vision system in MEW). The raw images (Supplementary Fig. 3) are processed in MATLAB to detect and measure the output features of the printing process (Supplementary Figs. 4, 5), including real-time data visualization.

The MEW hardware was characterized to confirm its functionality. Our testing showed that the automated MEW printer technology platform allows the user to alter input parameters and perform output measurements in real-time, which substantially accelerates data collection as well as provides the foundation for the implementation of closed-loop control functionality.

Algorithmic concept and system overview

This work seeks to establish a methodology for rapid experimentation and development of in-situ quality control of an MEW system using ANNs. The algorithm is based on supervised ML that trains a model of the system using data obtained through automated experimentation. Unlike numerical modelling methods, ANNs can be trained without a detailed understanding of the complex equations governing the dynamics of the system25. ANNs permit a highly nonlinear, electrohydrodynamic process like MEW to be modelled using experimental data and ML training. The model is subsequently used together with optimization to develop feed-forward and feed-back control, which allows input parameters for the desired output to be predicted, and output error to be measured and corrected, respectively. Consequently, automated experimentation combined with data-driven process modelling and control substantially reduces the complexity of the operation of the MEW printing process while improving the throughput and reproducibility of the system. Here, we present a proof-of-concept for using experimental data collection, model training, offline process simulation, and online quality control to rapidly develop an ML-driven feedback control process for MEW.

The overview of the system’s operation is summarized in Fig. 2. The process starts with the selection of the desired output parameters (Fig. 2A) such as resultant fibre size, deposition accuracy and printing speed, which refer to fibre diameter, jet angle and collection speed respectively (Supplementary Figs. 6–10). Additionally, the parameters can be prioritized to ensure that the most important objectives are reached (e.g., in some cases printing speed might be prioritized over deposition accuracy). The output selection is used by the ANN optimizer—a feed-forward data-driven model of the system—to predict the input parameter that will satisfy the requested output parameters (Fig. 2B). The simulated parameters are used as the initial input parameters to the system. As the printing progresses, the real-time monitoring system (Fig. 2C) provides a high-resolution video stream of the material deposition process (Fig. 2D). The acquired data is analysed and information about the output is extracted and displayed (Fig. 2E). The measured output is used by the ANN controller to compensate for the detected errors by adjusting the input parameters accordingly (Fig. 2B).

Fig. 2: Overview of the system functionality.
figure 2

A User selects printing properties such as the fibre size, deposition accuracy, printing speed and printing stability, which refer to fibre diameter, jet angle, collection speed and standard deviation of the Taylor cone area respectively. B User input is passed through the ANN optimizer—the feedforward model of the system which predicts a set of input parameters that will satisfy the user’s output parameter selection. C Printing commences using the predicted input parameters. The material deposition process is captured in real-time using the collimated backlight, telecentric lens, and monochrome camera. D Output images are collected and analysed. E Output data is displayed and used in (B) the feedback loop where the ANN controller is used to correct the discrepancy between the requested and measured output. F System architecture: MATLAB is used for data operations and user interface, while the microcontroller boards and camera are utilized for parameter control and output monitoring.

Full size image

The system architecture overview including parameter control, data acquisition and data processing is shown in Fig. 2F. MATLAB provides the computational setting for real-time input parameter control and output parameter measurement. Input parameters are controlled via serial communication between MATLAB and microcontrollers over USB, while measurement of output parameters is performed through analysis of the real-time video of the printing process. Each captured frame is analysed by the image processing algorithm where the Taylor cone area and jet angle are extracted and used to provide information about system stability, deposition accuracy, and resultant fibre size. More details about the image analysis algorithm can be found in Supplementary Figs. 2–10.

Automated experimentation and data collection

Automated experimentation was used to acquire information about the material deposition across different input parameter configurations. Throughout six separate printing jobs, 324 different combinations of input parameters were explored using a full factorial design of experiment. Each configuration of parameters was printed for 150 seconds to allow for process stabilization between parameter adjustments. More details on experimental data collection can be found in Supplementary Tables 1 & 2. The methodology was designed to provide robust process automation, increasing the throughput of the experimentation process.

Lastly, the relationship between the Taylor cone area and fibre diameter was established. While the literature shows a qualitative link between the two variables23, a quantitative correlation has not yet been developed. Here, using the automated MEW platform, several experiments were conducted to determine the relationship between the Taylor cone area and fibre diameter. The obtained function maps the Taylor cone area at the given collection speed to the corresponding fibre diameter. This allows estimation of the fibre diameter in real-time, to the best of our knowledge for the first time. A detailed description of the experiment and analysis can be found in Supplementary Figs. 10–16.

Data analysis and model training

Figure 3A shows the mean jet angle at different voltage and pressure levels. Similarly, Fig. 3B demonstrates the Taylor cone area at varying voltage and pressure levels. In both examples, we noticed that increasing pressure causes the jet angle as well as the Taylor cone area to increase; similarly increasing the collector speed will increase the jet angle. In contrast, increasing the electric field decreases the jet angle and Taylor cone area. The visualization of the results allows the user not only to inspect the quality of the experimental data but also to gain a better understanding of the process by visually analysing patterns in the data. Finally, the data was organized into input and output matrices that were subsequently used for ANN model training. The input matrix consists of the input parameters such as applied pressure, applied voltage, collection speed, and applied temperature while the output matrix includes the mean Taylor cone area, mean jet angle, and standard deviation of the Taylor cone area.

Fig. 3: Experimental data and modelling.
figure 3

A The Taylor cone area measured for different combinations of voltage and pressure under constant collection speed and temperature. B The jet angle measured for different combinations of voltage and pressure under constant collection speed and temperature. C A polynomial function relating Taylor cone area to fibre diameter ratio decreases with increasing collection speed. Graphical representation of the trained neural network model of (D) jet angle (E) the Taylor cone area, and (F) fibre diameter at the collection speed of 800 mm/min and temperature of 75 °C.

Full size image

The relation between the Taylor cone area and fibre diameter is shown in Fig. 3C. The decreasing Taylor cone area to fibre diameter ratio with increasing collection speed means that the fibre diameter is decreasing with increasing collection speed. This confirms the strong influence of the drag forces exerted on the fibre jet by the moving collector plate, causing fibre stretching and thus diameter reduction.

The ANN model of the MEW system was trained using ML algorithms via the MATLAB deep learning toolbox. The results of model training were tested against the test set consisting of 15% of the experimental data. A search was conducted to find the best performing combination of training algorithm, number of hidden layers, and number of hidden-layer neurons. A feedforward network consisting of one input layer with 4 neurons, one hidden layer with 12 neurons and one output layer with 3 neurons was trained using Bayesian regularization backpropagation (BRB), Levenberg-Marquardt, and scaled conjugate gradient optimization algorithms. Despite slower training time than Levenberg-Marquardt, BRB resulted in the best performance and was used to train the model. The obtained ANN was tested by performing additional test prints, where the ANN model was used to predict the output of the system given the input parameters. Although the present model considers only the most important process parameters, the ANN approach and highly digitalized platform allows easy scale up of the solution by including more variables into the model as well as updating the existing weights via further network training using different or additional datasets. Examples of the trained ANN model of the Taylor cone area, jet angle, and fibre diameter are shown in Fig. 3D–F, respectively. Supplementary Figs. 17–19 include more details about network training and testing.

Feed-forward process simulation

The trained feedforward model is used to forecast the expected output characteristics based on the given input parameters. However, the inverse of such a system is more advantageous as it allows the user to specify the desired output and obtain a prediction for the required input parameters to realize the desired output26. To achieve this, optimization methods were used in combination with the feedforward MEW model to develop functionality that minimizes a cost function under specified constraints. The cost function to be optimized is defined by the user as explained in Fig. 2A. The user’s specification of the output parameters and their importance are used to define the setpoints and constraints for the feedforward model which creates the cost function space. Here, optimization is used to minimize the cost function, which finds the most suitable set of input parameters that is predicted to realize the desired output. Such functionality can be used as an open-loop control system where the predicted input parameters are used and unchanged throughout the entire printing process to fabricate desired structures or as a simulation to check whether the requested output characteristics are achievable25. The section including Supplementary Fig. 18 has further details about the open-loop system design and operation.

Figure 4 shows multiple examples of the ANN model-based simulation for user-specified output parameters. Figure 4A–C, E as well as Fig. 4C–F show examples of the cost functions for requested collection speeds of 2000 mm/min and 800 mm/min, respectively. For all cases, the polymer temperature is 75 °C. Figure 4A shows the cost function for the requested diameter of 6 µm, while Fig. 4B represents the cost function surface for the requested jet angle of 35°. Lastly, Fig. 4C shows the function when both fibre diameter and jet angle setpoints are considered. Similarly, Fig. 4D and E displays the cost function for a requested fibre diameter of 14 µm and jet angle of 20°, respectively. The combination of requesting both output parameters at the same time is shown in Fig. 4F.

Fig. 4: Cost function landscape for feedforward control.
figure 4

Several examples of simulations and their corresponding cost function landscapes at the collection speeds of (A, B, C) 2000 mm/min and (D, E, F) 800 mm/min with polymer temperature of 75 °C. A Requested fibre diameter of 6 um, B Requested jet angle of 35° (C) Requested fibre diameter of 6 µm and jet angle of 35°, together. D Requested fibre diameter of 12 µm, (E) Requested jet angle of 20° and (F) Requested fibre diameter of 12 µm and jet angle of 20° together.

Full size image

Feed-back process control

Although the feedforward model provides a way to simulate the process and predict input parameters for the desired output, open-loop operation is usually not reliable due to a number of external factors, such as material degradation over time and fluctuating ambient temperature that are not included in the model. These uncertainties can cause errors that cannot be eliminated with an open-loop system. Here, a feedback controller was developed in order to compensate for the unmodelled parameters. The complete closed-loop control system uses the previously described feedforward model to compute initial input parameters while real-time output measurement provides feedback information that is compared with the requested output. The feedback controller is based on the Jacobian of the feedforward model that allows adjustment of the input parameters so that the desired output is reached and maintained. Although the fibre diameter cannot be directly measured during the printing process, the established correlation between the fibre diameter and Taylor cone area allows for accurate fibre diameter estimation and control. Supplementary Fig. 19 includes details about the design and operation of the closed-loop functionality.

Figure 5 shows several examples of the closed-loop operation of the system. The initial input parameters were chosen as suggested by the simulations shown in Fig. 4 and ran for 350 s to stabilise the printing process. After stabilisation, the closed-loop controller engages every 150 s to adjust input parameters in real-time. The target fibre diameter of 6 µm at 2000 mm/min and 12 µm at 800 mm/min are presented in Fig. 5A and Fig. 5B, respectively. Similarly, the jet angle results for the requested angle of 35° at 2000 mm/min and 20° at 800 mm/min are shown in Fig. 5C and Fig. 5D, respectively. Lastly, Fig. 5E, demonstrates the response of the system when both the fibre diameter and jet angle values are specified by the user. Although in both cases the simulation implied the requested output was not achievable for the given collection speed, the closest alternative was suggested. Thus, Fig. 5E shows the results for the requested fibre diameter of 4.9 µm and jet angle of 38° at 2000 mm/min, while Fig. 5F demonstrates the results for the requested fibre diameter of 9.2 µm and jet angle of 27° at 2000 mm/min. The data presented in Fig. 5 includes some minor fluctuations that are primarily due to the “back and forth” motion of the MEW process, where the jet  changes direction at every turn. While this jet information at the tool path turn points is filtered with an algorithm, it still results in some noise. Furthermore, frames that have blur due to the motion can be addressed in the future with stroboscopic image capture. Despite these fluctuations, in all presented examples, the system was able to successfully adjust the printing parameters in real-time to achieve the target output within 5% target error. Manual measurement of the fibre diameter was performed (offline via microscopy) to confirm the Taylor cone area to fibre diameter correlation. The measured data shows accurate fibre diameter prediction based on the Taylor cone area at a given collection speed.

Fig. 5: Closed-loop control results.
figure 5

Several examples of the closed-loop system operation with the specified request for (A, B) the Taylor cone area, (C, D) jet angle and (E, F) both the Taylor cone area and jet angle combined. In all examples, the initial system parameters are obtained through the feedforward model process simulation. After the initial system stabilization period of 350 s the closed-loop controller is engaged. The collected data is analysed, and the controller performs adjustments to the process parameters every 150 s. The results show the ability of the system to make corrective adjustments to maintain the desired output of the system within 5% tolerance of the requested output.

Full size image

Conclusion

3D printing technologies such as selective laser sintering, stereolithography and fused deposition modelling are successfully used in industries such as automotive, aerospace, health care etc. This is possible because detailed knowledge is available about processing properties and their complex relationships. Acquiring this knowledge, however, took decades using an iterative approach. With the MEW jet containing highly useful information for process control, we applied ML on camera vision datasets. A data-driven modelling approach was used in combination with automated MEW hardware to develop a methodology for the first closed-loop process control system for MEW. We found that the telecentric vision system and automated experimentation can be used to rapidly obtain a large amount of data characterizing the input-output parameter space to provide better process understanding through data visualization and modelling. The ANN algorithm with 12 hidden-layer neurons provides a suitable architecture for the presented experimental data. The model is used in an open-loop form to predict input parameters to realize user-specified output parameters. Lastly, the ANN-based closed-loop controller was developed to complement the feedforward functionality and vastly reduce measured output error in real-time. It was shown that the data-driven closed-loop system is able to reach the desired output (i.e., fibre diameter, jet angle, Taylor cone area) within 5% of the requested target across several experiments.

Real-time process monitoring and automated experimentation can be used with ML algorithms to successfully model a multidimensional, complex 3D printing system such as MEW and develop real-time quality control of the process. This approach decreases experimentation time while improving the throughput and fidelity of the printing process. The ML-based solution reduces the complexity of operation of the system, potentially allowing non-experienced users to print high-quality structures while improving reproducibility. In the future, the system and methodology could be used in real-time process quality control and rapid characterization of materials made via MEW. The system demonstrated here is an important technological advance to move the MEW technology to a higher technology readiness level and ultimately to become an industry-ready additive manufacturing technology.

Methods

MEW automated device

The MEW device used in this study was developed in accordance to the open-source assembly instructions provided in our previous study27 which includes all necessary details needed to build the automated printer. Additionally, MATLAB was used as a platform to develop algorithms encompassing data processing, user interface, model fitting, image processing, and real-time process control.

Optics and image analysis algorithm

Telecentric optics are used to measure the Taylor cone area, jet angle, and jet diameter. Here, the machine vision system consists of a telecentric lens (1.0 × 2/3” GoldTL™, Edmund Optics, USA), monochrome USB 3.0 camera (acA2440-75um ace, Basler, Germany) and collimated backlight (2” × 2” White Collimated LED Backlight, Metaphase Technologies, USA) as shown in Supplementary Fig. 1. The output image shows a well-defined silhouette of the observed object (Supplementary Fig. 2a) as compared to the previously used camera (Supplementary Fig. 2b). An image analysis algorithm was developed to analyse the images captured during the MEW material deposition process. The algorithm focused on the measurement of 3 output features: Taylor cone area, jet angle, and fibre diameter. The algorithm processed the data in real-time analysing each captured frame (Supplementary Fig. 3). An initial threshold was applied to the raw image in order to obtain a silhouette of the nozzle, Taylor cone, and fibre jet. Specifically, Otsu’s method28 was used to obtain a binary image of the captured frame followed by cropping to remove a fragment of the printhead visible in the top section of the image. The image was used to locate the nozzle tip and to compute the jet angle.

Jet angle and diameter calculation

The jet angle was computed by tracing the centre of the polymer jet from the previously found centre of the nozzle tip to the 60% of the working distance defined as the vertical distance between the nozzle tip and collector plate (Supplementary Fig. 6). The distance was chosen based on the previously published study23 that showed the correlation between jet angle measurements computed at different points along the jet.

The high-resolution camera used in this study allowed, for the first time, the measurement of the fibre jet thickness. The position along the fibre jet where the jet diameter is to be measured needed to be defined. We expected that the jet diameter would decrease as it gets closer to the deposition point. To demonstrate this, the diameter was measured at many different points along the fibre jet from the nozzle tip to the deposition point (Supplementary Fig. 7). The results not only confirmed that the jet diameter decreases during its flight between the nozzle and the collector plate, but also showed that the rate of decrease slows down as the jet approaches the collector plate (Supplementary Fig. 8).

Taylor cone area calculation

The Taylor cone area was measured by defining the bottom of the Taylor cone in the collected images. The height of the horizontal plane defining the transition between Taylor cone and fibre jet depends on system parameters. Based on a previous study23 the relative variation of the Taylor cone area is the same regardless of the definition of the cone endpoint, however, the resolution of the measurement improves with larger Taylor cone end definition. Therefore, the Taylor cone endpoint was defined as the fixed distance of 300 pixels from the tip of the nozzle.

Neural network training and design

The process of experimental data collection is described in Supplementary Information, with the data saved in csv format. Before the data is passed as an input-output training set, it was scaled and normalized. Neural network training was performed using MATLAB 2020b and the Deep Learning Toolbox. The hyperparameters for training were as follows: 3 different training algorithms (Bayesian Regularization Backpropagation, Levenberg–Marquardt, and Scaled Conjugate Gradient), and 4 different numbers of neurons in the hidden layer (4, 8, 12, 16) were tested. Each network was trained 5 times. The shallow neural network architecture with 1 input layer, 1 hidden layer consisting of 12 hidden neurons, and 1 output layer was chosen. It was found that Bayesian Regularization Backpropagation was the best performing optimization algorithm. Open-loop and closed loop control models are described in the Supplementary Information.