ArticlesAll Issue
ArticlesMagnetic-Actuated Cyber-Physical System for Interventional Surgery
• Chudong Shan1 , Jianhui Zhao1,* , Wenyuan Zhao2 , Tingbao Zhang2 , Bo Du1 , and Zhiyong Yuan1,*

Human-centric Computing and Information Sciences volume 12, Article number: 37 (2022)
https://doi.org/10.22967/HCIS.2022.12.037

Abstract

At present, interventional surgery has been successfully applied in the treatment of many diseases, since due to the significant dosage of ionizing radiation, surgeons and staff are susceptible to serious health problems. Nowadays guidewires and catheters are not capable of active steering by themselves remotely, so achieving precise control of the movement of the guidewire in interventional surgery is a hot issue at the forefront of research in related fields. In this paper, we propose a cyber-physical system that consists of a magnetically actuated patient module and operator module with a virtual surgical environment. Unlike the existing magnetic-actuated systems, our system adopts dynamically changing currents through coils to excite a dynamic magnetic field, which results in more flexible and rapid control. The experimental result shows that our method can control the direction of a magnetic field vector within an average absolute error of 2° and an average error of –0.127°, and the single-round calculation time of an electric current configuration is only 0.2 ms, which meets the requirement of real-time guidance for catheter intervention.

Keywords

Cyber-Physical System, Human-Computer Interaction, Magnetic Actuation, Real-Time Visualization, Surgical Simulation

Introduction

Interventional surgery is an important way to diagnose and treat heart disease by introducing special catheters, guidewires, and other precise medical instruments into the human body under the guidance of medical imaging equipment (such as digital subtraction angiography) [1], which is minimally invasive, effective, and has few complications. Since cardiovascular disease is currently the leading cause of human mortality worldwide [2], the construction of remote interventional diagnosis and treatment systems also has been a main focus of comprehensive cross-field development, and thus it will have a profound impact and important value in solving the problem of uneven distribution of resources for interventional diagnosis and treatment of complex cardiac diseases. In addition, the extensive development of robotics, computer technology, and communication technology has extensively promoted the application of related technologies in interventional surgery [3].
However, interventional surgery also has obvious drawbacks as follows:
(1) Medical imaging navigation equipment will emit X-rays, while surgeons and staff will be injured when exposed to the significant dosage of ionizing radiation [4].
(2) A doctor's skill and ability to perform interventional surgery largely depend on experience in surgical practice, although training a qualified doctor is difficult and costly, and training based on live animals has bioethical issues [5].
Using computer-assisted methods is considered as a promising, viable solution, which can provide interventional surgeons with enhanced flexibility and visualization, and tactile perception, improve the consistency of surgery, eliminate the difficulties of traditional surgery and interventional instruments, and improve the safety of interventional surgery [6, 7]. Compared with the catheters or guidewires operated by doctors, the computer-assisted systems have an active guiding ability. There are currently two dominant methods of guidewire manipulation geared toward interventional surgery, namely mechanical and magnetic control methods. The shortcoming of the mechanical method is that it is difficult to achieve precise control of the slim guidewire using mechanical structures due to the limitation of the catheter diameter to millimeters. At present, researchers have studied the application of magnetic fields in interventional surgery and demonstrated the feasibility and broad application prospects of the magnetic control method. There have been promising research advances in the study of manufacturing technology and physical software simulation [8]. Therefore, the magnetic control method is considered to be another more promising solution for applications. However, the traditional permanent magnet drive method does make it difficult to achieve flexible control, and the application scenario is limited.

Fig. 1. (a) Screenshots of 3D reconstruction scene of catheter intervention in perspective and cross-sectional views. Enhanced device visualization can provide real-time visual feedback. (b) An operator interacting with a magnetic-actuated virtual catheterization system. Green represents interacting with a virtual scene using a gamepad; yellow, a coil array steering the guidewire with an RGB camera as a visual input device; red, the embedded module receives current configuration and performs current output; and blue, the simulated surgery scene is rendered and presented on the screen.

To solve the problems in the above existing methods, we propose a magnetically controlled method for driving a catheter through a coil array, where the control current through the coil array is calculated using the fast power-optimal method, which enables the coil array to generate a suitable magnetic field to drive the catheter (refer to Fig. 1). Unlike a static magnetic field generated by permanent magnets, a dynamic magnetic field generated by changing electric current flow requires real-time calculation. The latter only needs to change the current configuration, which has better control flexibility and response speed than a permanent magnet with a static magnetic field [9], since steering the guidewire with a static magnetic field requires using a mechanical device to adjust the posture of the permanent magnet. To perform magnetic field calculation, mature commercial software is currently available, such as ANSYS Maxwell and COMSOL [10]. These well-developed software can accurately calculate the magnetic field generated by the coil array of any topology structure on the model grid to analyze its magnetic field characteristics. However, they are all based on discrete numerical methods such as finite element analysis and finite difference methods that require a large amount of computation [11]. generated by the coil array of any topology structure on the model grid to analyze its magnetic field characteristics. However, they are all based on discrete numerical methods such as finite element analysis and finite difference methods that require a large amount of computation [11].
The main contribution of this article is to start from the dynamic mechanics' principles of the magnetically controlled guidewire, and propose fast algorithms for the calculation of a magnetic field and control current as follows:

We propose a novel magnetic-actuated catheterization device for guidewire steering in a remote manner based on continuum mechanics and electromagnetic principles. The device uses a dynamically changing magnetic field, which makes the control faster and more flexible.

We propose a geometric method to describe the configuration space of a magnetic field vector, which directly determines the motion control range of the guidewire, and discuss the control capabilities and limitations of the system.

We propose a real-time algorithm for controlling current calculation and prove mathematically that this method optimizes the overall thermal power of the coil array.

The goal of this paper is to propose a magnetic-actuated cyber-physical system (CPS) for interventional surgery. The remainder of this paper is arranged as follows. In Section 2, we provide a literature review on existing interventional surgical systems and corresponding methods they use. In Section 3, we present the overall architecture of the proposed interventional surgical system, as well as the methodology and derivation of the magnetic field and current configuration during interventional surgery. In Section 4, we present the hardware configuration and materials used along with the specific parameters of the software, and finally, we derive and analyze the experimental results. Section 5 provides the paper’s conclusion and a future outlook.

Related Work

There are already some kinds of interventional surgical systems introduced, and such systems have utilized a master-slave architecture [1214]. In general, there are two mainstream solutions of the mechanical method and magnetic control method [15, 16].

Mechanical Method
Beyar et al. [17] introduced a remote navigation system that involves a computer-controlled wire and delivery system navigator. This system consists of an operator module equipped with a touch screen monitor, joystick as the input device, and patient module with a guidewire actuated mechanically. The guidewire is maneuvered using both a joystick and touch screen, while axial and rotational guidewire motions are achieved by a mechanical transmission module. Starting from the first principles of solid mechanics, Camarillo et al. [18] introduced the mechanical model of a tendon-driven continuous manipulator and developed a steerable cardiac catheter based on that. Due to the limitation of the diameter of the catheter in millimeters, it is tough to achieve precise control of the guidewire using a mechanical structure. Therefore, magnetic control is considered another promising solution.

Magnetic Control Method
As for the magnetic control method, Ernst et al. [19] developed a remote-control robotic catheterization system for electrophysiological mapping and radio-frequency ablation. It consists of 2 computer-controlled permanent magnets located on opposite sides of a patient, and its motor drives the catheter back and forth, which is proved to be a safe and feasible tool for remote catheter ablation of AV nodal reentrant tachycardia (AVNRT). Yang et al. [20] proposed a miniature magnetically drilled tip guidewire with controlled bending in a directional magnetic field that can navigate through complex vasculature and is magnetically controlled with an integrated drive system. Compared with the mechanical method, the magnetic-driven guidewire is more flexible and can be manufactured to be smaller than a coronary artery [21]. Since interventional surgery requires increasingly flexible and smaller catheters designed to ensure deeper penetration into a patient through the vasculature, it has become infeasible to control a catheter using mechanical methods [22].

Magnetic Control Method
In addition to the above two methods, many scholars have investigated other methods such as the chemical actuation [23, 24] and optical actuation methods [25, 26]. However, the chemical actuation method may cause biological incompatibility and make it difficult to achieve precise interventional control [21], while the optical actuation method cannot penetrate deeply into non-transparent media and deeper areas of the human body [27]. Therefore, this paper will focus on the magnetic control method.

Materials and Methods

Architecture
Even though the current interventional surgery technology is very advanced, it is still very challenging to perform interventional surgery completely by computer or robot independently, and using master-slave architecture can fully combine human intelligence with superior mechanical performance, the majority of surgical robots designed and implemented to date use a master-slave architecture [28]. Similar to other master-slave architecture which consists of master and slave modules, the CPS we designed consists of two modules, namely an operator module with a virtual surgical environment, and a magnetically actuated patient module, both of which use the wireless network to communicate [2934]. The operator module displays the virtual scene of the interventional surgery on a screen and accepts user input from devices such as a joystick. We also propose a real-time calculation method of the magnetic field and fast power-optimal algorithm of the control current based on physical principles, which are the core algorithms of our system. The patient module comprises a coil array, controlled guidewire, visual input device, edge computing device, and embedded control device. And the operator module's virtual surgical environment is a program constructed based on computer graphics technology and real-time physical simulation methods. The system is illustrated in Fig. 2 and has a closed-loop control flow denoted by purple arrows and detailed in Fig. 3. The user specifies the spatial target position x ̃ that the front end of the guidewire is expected to reach. Then, the front end of the guidewire’s position changes, and the visual input device captures an image to obtain the new position x, thus providing feedback on the input to correct the next round of the control process. The input of the solver is the difference δ ̃ between the target position and visually captured position, and the output is a list of control current that each coil in the coil array should flow through. The appropriate electric current configuration is sent to the embedded module for output. The essence of the solver is to solve the inverse process of a process that occurs in the physical environment [35]. The variables (here X represents any variable) in the solver satisfies the same form of equation as the variable X in the physical environment. We use mathematical language to express this fact as implicit Equations (1) and (2), where f and g are functions that can be determined and are elaborated in the coming subsection:

(1)

(2)

Fig. 2Flowchart of our system for magnetic-actuated catheterization and visualization. The patient module contains image capture devices and an embedded module that adjusts the output current, and the doctor module is for remotely sending instructions and visualization.

Fig. 3Flowchart of the closed-loop control system. Included are the inputs, solvers, and actuators denoted with purple arrows in Fig. 2. The solver and actuator refer to the current solver and embedded module in Fig. 2.

Physical Principles
Magnetic controlled guidewires are generally designed and manufactured with permanent magnet materials, such as NdFeB [36]. The front-end part of the guidewire is magnetized before use and has polarity along the centerline. In an external magnetic field, the flexible front end of the guidewire is simultaneously subjected to magnetic Cauchy stress, resilient elastic force, and gravitational force due to magnetic field, deformation, and gravity. Qualitatively speaking, the force direction of a magnet in a magnetic field is related to the direction of the magnetic field vector. The force direction at the N pole of the magnet is the same as the direction of the magnetic line of force, and the force direction at the S pole is opposite to it.

(3)

By solving the physical Equation (3) in the equilibrium state, it can be known that the bending distance δ of the front end of the guidewire and the magnetic field vector B at the same spatial point approximately satisfy the proportional relationship [37]:

(4)

where M is the magnetization of the magnet at the front end of the guidewire. Furthermore, because the magnetic field excited by the coil array is far from the intensity required to magnetize the permanent magnet, the magnitude of magnetization M can be regarded as a constant, so Equation (4) can be further simplified as:

(5)

where θ is the angle between B and M; δ is perpendicular to M; δ, B, and M are coplanar. Although the magnetic field is never a constant vector field in practice, it can be regarded as uniform locally.

Real-Time Computation of Magnetic Field
Although discrete numerical methods on a grid can solve the magnetic field of any material on any geometric shape, researchers have shown that in many cases, the analytical solution of regular geometric shape or the piecewise linear analytical solution can be used to provide a good approximation within an acceptable error range [38].
Let’s consider a thin solenoid with a height of 2b and radius of a, and has n turns per unit height, with an electric current of intensity I flowing through it. The total current intensity on the surface is Its center is placed at the origin of the coordinate system, and the axial direction coincides with the z-axis. Such a coordinate system is called a local coordinate system of the solenoid. In a cylindrical coordinate system, the magnetic field can be expressed as which is similar to the cartesian coordinate system The transformation equation between these two expressions is expressed as:

(6)

The analytical solution of the magnetic field B around an ideal solenoid can be expressed as a double integral form according to the Biot-Savart law [39] as follows:

(7)

Analytical results of this integral expression are as follows:

(8)

(9)

(10)

The cel function (Equation (11)) is the generalized complete elliptic integral, and degenerates into regular complete or incomplete elliptic integrals by special values of function parameters. As for program implementation, we use the Bulirsch algorithm to perform fast numerical calculations [40].

(11)

The coil array is composed of several coils. According to the superposition principle of the magnetic field, a magnetic field vector at a certain point in space is the vector summation of the magnetic field vectors at the same point excited by all sources. Note that the magnetic field induced by the i-th coil at position x is $B_i$ in the array’s local coordinate system; and $T_i$ and $R_i$ are the translation transformation matrix and rotation transformation matrix from i-th coil’s local coordinate system to the array’s local coordinate system; $x_i$ is the representation of the corresponding spatial position in i-th coil’s local coordinate system, then equation x=$T_i$ $R_i$ $x_i$ satisfies. The magnetic field B induced by the coil array at x is the summation of $B_i$:

(12)

Configuration Space of Magnetic Field Vector
For a coil array composed of N stationary coils, the magnetic field inducted at the spatial position r is B(r), and magnetic field inducted by the i-th coil at the same position is denoted as $B_i$(r), which according to Biot-Savart’s law integration obtains the integral expression of $B_i$(r):

(13)

The part of the expression that does not contain I in the above formula is denoted as . Because the geometric parameters of the coils are fixed, is an explicit expression of a constant vector field, and is exactly the magnetic field excited by the i-th coil with unit current intensity. For the coils, we stipulate that the positive direction of current flow moves counterclockwise around the positive direction of the z-axis. A negative value represents a current flow with the same current intensity but in a clockwise direction. Let’s denote as the linear combination of the corresponding magnetic field when the current takes the upper and lower limits, while the upper and lower limits of the coil passing current are denoted as and , respectively:

(14)

For usual hardware implementations, the absolute values of and are identical. The sign of is positive, and the other is negative. Further discussion in this article will satisfy this restriction by default, expressed in mathematical language, Equation (15) is always satisfied. Alternatively, according to the superposition principle of the magnetic field, the total magnetic field vectors B(r) and $B_i$ (r) inducted by a single coil at the same spatial position r satisfy the relationship

(15)

Here, we define as the linear combination of is the set of magnetic field vectors that can be obtained by configuring specific currents flowing through each coil at r. Ω_B (r) is called the configuration space of B(r) of the coil array at r. Alternatively, from a geometric point of view, the vector set includes all the upper and lower limits of the magnetic field vector at the space point r. The power set of is the set formed by all the subsets of $S_r$. The number of elements in the power set are . Next, we define a class of functions σ:{x_i |x_i∈R^3}→R^3, which are mappings from a set of vectors to a point in space:

(16)

Here, we present a theorem to further compute the configuration space of a magnetic field vector using the quick hull algorithm [41].
Theorem 1. The convex hull CH(σ_r (P(S_r))) of the point set σ_r (P(S_r)) is equivalent to the configuration space of the magnetic field vector $Ω_B$(r) at r, that is, CH($σ_r$ (P($S_r$)))=$Ω_B$(r).
Fig. 4 shows the respective configuration space of a magnetic field vector by coil arrays of two, three and five coils at some point in the space, where the red marking points correspond to the elements in the set σ_r (P(S_r)). Literatures [42] and [43] give a fast algorithm for computing a convex hull from a set of points. It can be seen from Fig. 4 that the configuration space corresponding to a 2-coil array is a parallelogram. According to the physical principle of guidewire steering, the guidewire can only be bent and guided in a particular plane. And a coil array composed of three or more coils results in a Ω_B (r) of convex polyhedron shape, which can bend and guide the guidewire in any direction perpendicular to centerline direction. Therefore, a coil array composed of at least three coils is required to bend and guide the guidewire freely.
Fig. 4Illustrations of the convex hull of $Ω_B$ (r) at position r in space for (a) the 2-coil array, (b) 3-coil array, and (c) 5-coil array examples.

Power-Optimal Control Current Solver
Concurrently, we hope to reduce the overall thermal power of the coil array (P) as much as possible.

(17)

Concurrently, we hope to reduce the overall thermal power of the coil array (P) as much as possible.

(18)

If B^* (r) is in the configuration space of a magnetic field vector, that is, there is at least one solution such that. That is, there exists I, which satisfies the following:

(19)

Suppose B*(r) is outside of $Ω_B$(r), which is beyond the capability of the system. In that case, theoretically, it is impossible to find a perfect solution for the current configuration, so then our goal degenerates to find an approximate solution using optimization methods.
We present a theorem here before continuing to propose an algorithm for controlling a current calculation.
Theorem 2 ([44]). Given matrices A∈$C^{n,m}$ and b∈$C^{n,1}$, the singular value decomposition form of A is A=VΣW^H. The Moore–Penrose generalized inverse of A is defined as satisfies the following:

(20)

And $x^*$ has the smallest L2-norm while minimizing the square error ‖Ax-b‖ as follows:

(21)

Now we can revisit our actual control current configuration problem as follows:
(i) For the first case of $B^*$(r)∈$Ω_B$ (r), the configuration space of current vector $Ω_I$ has at least one solution, resulting in $B^*$(r) being generated by the coil array. An accurate numerical solution $I^*$ can be obtained by using the singular value decomposition (SVD) method according to Theorem 2.
be the SVD representation of , and its Moore–Penrose generalized inverse is Then satisfies inequality (refer to Equation (20)):

(22)

And according to Equation (21), $I^*$ has a minimal L2-norm among all Is meeting the requirement of Equation (19) as follows:

(23)

According to Joule’s law of electricity, the total heating power of the coil array is , because I*’s square of L2-norm is minimal, while the heating power P has the minimum value in tandem.
In summary, the total electric power of the coil array reaches its minimum while obtaining the desired magnetic field vector of a specific control point.
(ii) In the other case of B* (r)∉$Ω_B$(r), the expected magnetic field vector is outside the configuration space, so it is impossible to achieve our previous goal. We have to lower the target to obtain an approximate result in the same direction with its L2-norm as close as possible. Therefore, B* (r) is corrected to the vector corresponding to it’s intersection with the convex hull’s boundary ∂$Ω_B$(r). Based on the discussion of the above two cases, our power-optimal algorithm expressed in pseudocode (Algorithm 2) is as follows:

Fig. 5Prototype design of the guidewire's front-end structure.

4.1.2 Control current output
We adopt a pulse-width modulation (PWM)-controlled H-bridge current output module to perform the control current output. Each output channel has two pins to control the current output direction, and the duty cycle of PWM controls the output voltage. Because the resistance of the coil is fixed, the voltage at the output circuit end is proportional to the current. In the hardware implementation we adopted, the output voltage is roughly linear with the PWM duty cycle. Still, it is nonlinear under a tiny duty cycle, as shown in Fig. 6, so a correction must be made [45, 46].

The correction is represented as Equation (24):

(24)

where duty I. Then

(25)

Fig. 6Relationship between the output voltage of the H-bridge module and input PWM duty cycle. A negative duty cycle means that the reverse output pin setting is enabled. We assume that the output curve is a curve that crosses the origin (denoted by orange line) but performs an additional correction step on the microprocessor

Quantitative Verifications
Table 1 lists the physical parameter settings in the quantitative verification experiment. The coil array is the main body of our magnetron equipment, which is used to generate a changing dynamic magnetic field, while the guidewire is similar to the existing systems, with which a permanent magnet is embedded in the tip. Since this experiment is a principle verification experiment, the guidewire is a rubber hose that is thicker than the guidewire used in clinical practice.

 Object Parameter Value Coil Inner radius (mm) 13.5 Outer radius (mm) 35 Height (mm) 62 No. of turns 1,024 Resistance (Ω) 10 Material Copper Hose Inner radius (mm) 1.5 Outer radius (mm) 2.5 Material Silicone rubber Permanent magnet Radius (mm) 1.5 Height (mm) 9 Material NdFe35

4.2.1 Magnetic field calculation
Table 2 lists the time required to calculate the magnetic field in the space around a single coil using different methods. All our evaluation procedures run on an 8-core AMD Ryzen 7 5800H CPU. Among these methods, the numerical integration and analytical solution methods are suitable for a particular position in space. In contrast, the finite element method (FEM) must be calculated on the entire model grid. Due to calculating a large number of unnecessary sampling points, the actual real-time performance is far inferior to the numerical integration and analytical solution methods.
In our CPS’s control loop, real-time calculation of the magnetic field is required for a single control point, and the source of the magnetic field is a regular cylindrical coil shape. The comprehensive evaluation result shows that the analytical solution method is the best option for this application.

Table 2. Comparison of numerical calculation methods of magnetic field
 Category Method applied Elapsed time (s/point) Numerical integration [47, 48] Global adaptive 1.4 Monte Carlo 4.5 Finite element method FEM (ANSYS) Analytical solution Ours (C++ implementation)

4.2.2 Magnetic field calculation
We design a 4-coil array, with all four coils placed upright and closed connected, while the centers of the coils are connected to form a square (as shown in Fig. 7). A PWM-controlled H-bridge module is adopted to output the control current (refer to Section 4.1.2), and an ESP32 microcontroller is used as the core controller of this module.
Fig. 7Design of 4-coil array.

Fig. 8Diagram of magnetic field vector’s configuration spaces $Ω_B$ (0,0,5), $Ω_B$ (0,0,35) and $Ω_B$ (0,0,70). $Ω_B$ (0,0,5) as too flat and sharply shaped. The former are not conducive to controlling the bending of the guidewire in the vertical direction; and the latter $Ω_B$ (0,0,70) will cause difficulties in magnetic guidance control in the horizontal direction.

Next, we compute the convex hull and visualize $Ω_B$(0,0,z), which is the configuration space of the magnetic field vector on the z-axis. The resulting graphics (refer to Fig. 8) show that the optimal control range of this coil array is 25–50 mm above the top of the coil. The shape of the configuration space within this range is relatively close to a sphere, which is suitable for controlling bending everywhere. If the shape of $Ω_B$ is too flat, it will be difficult to bend the guidewire vertically; conversely, if the shape is too sharp, it will be difficult to control the horizontal bending of the guidewire.
To verify the correctness of the mathematical model proposed in Section 3.1, we place a compass in the optimal control area to show the direction of a magnetic field vector, specify magnetic field vectors, and set the compass point to the desired direction in the horizontal plane. Firstly, we set a magnetic field vector that forms a specific angle with the positive x-axis, and then measure the angle formed by the deflection direction of the compass and positive x-axis. The experimental phenomena indicate that the direction of a magnetic field vector implied by the compass is the same as the specified direction.
We set the direction of a magnetic field vector to be generated every 5° from 0° to 360°, and measure the deflection angle of the compass. The quantitative experimental results are shown in Fig. 9. The polar graph on the left shows the error between the compass rotation angle and a specified angle, while the error $E_θ$ is distributed between –6° and 6°. On the right is a box whisker graph showing the error distribution. Most of the errors concentrate between –1° and 1°, with the median specified as 0°, mean error as only –0.127° and mean absolute error as 2°. In terms of real-time performance, on an 8-core Ryzen 7 5800H CPU, the CPU time required for a single calculation of control current configuration is less than 0.2 ms.
Within the allowable error range, the experimental results verify the correctness of the theoretical method described in Section 3.1, and the algorithm implementation meets the real-time requirement.

Fig. 9Accuracy experiment result graph. The error between the angle of compass rotation and a specified angle:

Guidewire Steering
Using the same experimental setup described in the previous subsection, we demonstrate the primary capability of our system designed for catheterization, based on active steering upon the magnetic actuation. We put the permanent magnet part of the guidewire tip in the optimal control area. The solver will calculate an electric current configuration to excite a magnetic field according to the operator’s bending direction instructions. The entire process is highly automated, so the operator only needs to specify the direction of bending with an input device such as a keyboard or joystick. The control loop flowchart is shown in Fig. 3. The basic principle of magnetic actuation and guidewire steering is to point the direction of the required magnetic field vector to the bending direction, thereby allowing the magnetization direction of the permanent magnets at the guidewire tip to align in the required direction, and finally causing the guidewire to bend towards that direction. Concurrently, we use the 4G network and MQTT protocol to realize wireless communication and control between the operator and patient modules under a local area network, which lays the foundation for future applications of remote interventional surgery.
As shown in Fig. 10, the tip of the guidewire is accurately and quickly pointed in the desired direction. As for the deformable guidewire modeling, we first define a set of 3D key points, then use a $C^2$ continuity interpolation function to obtain the centerline from the key points, and use a parallel transport approach to frame the centerline curve [49]. Lastly, we triangulate the model and send the data to a GPU rendering pipeline for visualization, as illustrated in Fig. 2.

Fig. 10Snapshots of the guidewire steering experiment. (a) The cross-sectional view of the blood vessel model and deformable virtual guidewire model in the virtual scene. (b) The corresponding physical entity scene captured by our system’s RGB camera, showing that the guidewire is steering under the applied magnetic field.

Conclusion and Future Work

Hardware Implementation Details

In our work, starting from the physics-based dynamics principles and continuum deformation theory, we have proposed a magnetic-actuated catheterization CPS based on a power-optimal control current solver which meets the real-time requirement. Notably, a general control method that has little to do with the topology of the coil array is proposed, and a control current solver is developed based on this. We also discussed the theoretical limitation of the magnetic field vector excitation capability through a geometric approach, which is directly related to the control capability of our system. What’s more, the solver integrates a fast magnetic field computing algorithm based on known analytical solutions, which is 1 to 6 orders of magnitude faster than the existing mainstream discrete numerical calculation method such as FEM. And the solver minimizes the overall heating power of the coil array, and outputs an appropriate control current configuration to steer the guidewire. The experimental results also show the high consistency between our model and the physical world.
However, our experiment is still in the prototype design stage, and there is still a long way to go before clinical verification. There is still a lot of improvements to be made in the material of the guidewire and design of the coil array, among other aspects. Our future work will mainly involve a magnetic field source, controlled guidewire, and development of a more advanced visualization environment. As for magnetic field sources, we will continue to explore the actual application of the algorithm in more complex coil configurations such as a 6-coil array or even 8-coil array. Also, we will study the influence of the material and structure design of the guidewire on the control accuracy and maximum deflection angle. Lastly, migrating 3D visualization from flat vision to a more cutting-edge hardware platform with stereo glasses or helmets would provide a more immersive virtual environment [50, 51]. In addition to this, we have only achieved local area network control with the 4G network, and in the future, we will conduct experiments using the 5G network and conduct clinical trials of remote interventional surgery when conditions are ripe.

Acknowledgements

Portions of this work were presented at the 8th International Conference on Control, Automation and Robotics in 2022, “Real-time Magnetic-Actuated Control and Virtual Interaction for Remote Interventional Surgery.” This manuscript expands significantly on our initial conference paper.

Author’s Contributions

Funding acquisition, ZY, JZ. Writing of the original draft, CS. Technical guidance, WZ, TZ. Writing of the review and editing, BD.

Funding

This work was supported by the Natural Science Foundation of China (Grant No. 62073248), the Translational Medicine and Interdisciplinary Research Joint Fund of Zhongnan Hospital of Wuhan University (No. ZNJC201926), and the Science and Technology Major Project of Hubei Province (NextGeneration AI Technologies) (No. 2019AEA170).

Competing Interests

The authors declare that they have no competing interests.

Author Biography

Name : Chudong Shan
Affiliation : School of Computer Science, Wuhan University.
Biography : Chudong Shan is currently studying for a Master of Engineering degree from the School of Computer Science, Wuhan University. He received his B.E. degree from the software college of Northeastern University (China) in 2020. His research interests include human-machine interaction, virtual reality, and embedded systems.

Name : Jianhui Zhao
Affiliation : School of Computer Science, Wuhan University.
Biography : Jianhui Zhao received the B.S. degree in computer engineering from Wuhan University of Technology, Wuhan, China, in 1997, the M.S. degree in computer science from the Huazhong University of Science and Technology, Wuhan, in 2000, and the Ph.D. degree in computer science from the Nanyang Technological University, Singapore, in 2004. He is currently an Associate Professor with the School of Computer Science, Wuhan University, Wuhan, China. His research interests include computer graphics, image processing, and machine learning.

Name : Wenyuan Zhao
Affiliation : Zhongnan Hospital of Wuhan University
Biography : Wenyuan Zhao is a chief physician and associate professor at the Zhongnan Hospital of Wuhan University. He is mainly engaged in the diagnosis and interventional treatment of cerebrovascular diseases, and has conducted in-depth research and treated a large number of clinical cases in intracranial aneurysms, cerebrospinal vascular malformations, dural (spinal) arteriovenous fistulas, and cerebral blood supply artery stenosis.

Name : Tingbao Zhang
Affiliation : Zhongnan Hospital of Wuhan University
Biography : Tingbao Zhang is an attending neurosurgeon and deputy director of the Interventional Center for Cerebrovascular Surgery at the Zhongnan Hospital of Wuhan University. He has been engaged in clinical work of neurosurgery for more than 10 years, specializing in the microscopic and interventional surgical treatment of neurosurgical cerebrovascular diseases, especially the surgical and interventional treatment of ischemic cerebrovascular diseases, and has rich clinical experience. He has published several papers in domestic and international professional journals.

Name : Bo Du
Affiliation : School of Computer Science, Wuhan University.
Biography : Bo Du is currently a professor and the Dean of the School of Computer Science, Wuhan University. He received his Ph.D. degree from the State Key Lab of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University in 2010. His research interests include pattern recognition, hyperspectral image processing, machine learning, and signal processing.

Name : Zhiyong Yuan
Affiliation : School of Computer Science, Wuhan University.
Biography : Zhiyong Yuan is currently a professor at the School of Computer Science, Wuhan University. He gained a Ph.D. degree in control science and engineering from Huazhong University of Science and Technology. He received his B.S. degree in computer applications and an M.S. degree in signal and information processing from Wuhan University. His research interests include human-computer interaction, virtual reality, machine learning, and embedded systems.

References

[1] A. Hooshiar, S. Najarian, and J. Dargahi, “Haptic telerobotic cardiovascular intervention: a review of approaches, methods, and future perspectives,” IEEE Reviews in Biomedical Engineering, vol. 13, pp. 32-50, 2020.
[2] World Health Organization, “Cardiovascular diseases (CVDs),” 2021 [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds).
[3] S. B. Xia and Q. S. Lu, “Development status of telesurgery robotic system,” Chinese Journal of Traumatology, vol. 24, no. 3, pp. 144-147, 2021.
[4] Y. Zhao, Z. Mei, J. Mao, Q. Zhao, G. Liu, and D. Wu, “Remote vascular interventional surgery robotics: a literature review,” 2021 [Online]. Available: https://www.techrxiv.org/articles/preprint/Remote_Vascular_Interventional_Surgery_Robotics_A_Review/14994123.
[5] K. E. Al-Ebrahim, “The interventional cardiology and the cardiac surgeon,” Pediatric Cardiology, vol. 43, article no. 474, 2022. https://doi.org/10.1007/s00246-022-02821-3
[6] S. A. Menaker, S. S. Shah, B. M. Snelling, S. Sur, R. M. Starke, and E. C. Peterson, “Current applications and future perspectives of robotics in cerebrovascular and endovascular neurosurgery,” Journal of Neurointerventional Surgery, vol. 10, no. 1, pp. 78-82, 2018.
[7] X. Xiao, S. Zhao, Y. Meng, L. Soghier, X. Zhang, and J. Hahn, “A physics-based virtual reality simulation framework for neonatal endotracheal intubation,” in Proceedings of 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Atlanta, GA, 2020, pp. 557-565.
[8] W. Hu, G. Z. Lum, M. Mastrangeli, and M. Sitti, “Small-scale soft-bodied robot with multimodal locomotion,” Nature, vol. 554, no. 7690, pp. 81-85, 2018.
[9] S. Jeon, A. K. Hoshiar, S. Kim, S. Lee, E. Kim, S. Lee, K. Kim, J. Lee, J. Y. Kim, and H. Choi, “Improving guidewire-mediated steerability of a magnetically actuated flexible microrobot,” Micro and Nano Systems Letters, vol. 6, article no. 15, 2018. https://doi.org/10.1186/s40486-018-0077-y
[10] E. Madenci and I. Guven, The Finite Element Method and Applications in Engineering Using ANSYS. New York, NY: Springer, 2015.
[11] Y. M. Tang, A. F. Zhou, and K. C. Hui, “Comparison of FEM and BEM for interactive object simulation,” Computer-Aided Design, vol. 38, no. 8, pp. 874-886, 2006.
[12] H. Wang, J. Chang, H. Yu, H. Liu, C. Hou, and H. Lu, “Research on a novel vascular interventional surgery robot and control method based on precise delivery,” IEEE Access, vol. 9, pp. 26568-26582, 2021.
[13] Y. Zhao, H. Xing, S. Guo, Y. Wang, J. Cui, Y. Ma, Y. Liu, X. Liu, J. Feng, and Y. Li, “A novel noncontact detection method of surgeon’s operation for a master-slave endovascular surgery robot,” Medical & Biological Engineering & Computing, vol. 58, no. 4, pp. 871-885, 2020.
[14] X. Jin, S. Guo, J. Guo, P. Shi, T. Tamiya, and H. Hirata, “Development of a tactile sensing robot-assisted system for vascular interventional surgery,” IEEE Sensors Journal, vol. 21, no. 10, pp. 12284-12294, 2021.
[15] H. Alfalahi, F. Renda, and C. Stefanini, “Concentric tube robots for minimally invasive surgery: current applications and future opportunities,” IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 3, pp. 410-424, 2020.
[16] Z. Yang and L. Zhang, “Magnetic actuation systems for miniature robots: a review,” Advanced Intelligent Systems, vol. 2, no. 9, article no. 2000082, 2020. https://doi.org/10.1002/aisy.202000082
[17] R. Beyar, L. Gruberg, D. Deleanu, A. Roguin, Y. Almagor, S. Cohen, G. Kumar, and T. Wenderow, “Remote-control percutaneous coronary interventions: concept, validation, and first-in-humans pilot clinical trial,” Journal of the American College of Cardiology, vol. 47, no. 2, pp. 296-300, 2006.
[18] D. B. Camarillo, C. F. Milne, C. R. Carlson, M. R. Zinn, and J. K. Salisbury, “Mechanics modeling of tendon-driven continuum manipulators,” IEEE Transactions on Robotics, vol. 24, no. 6, pp. 1262-1273, 2008.
[19] S. Ernst, F. Ouyang, C. Linder, K. Hertting, F. Stahl, J. Chun, H. Hachiya, D. Bansch, M. Antz, and K. H. Kuck, “Initial experience with remote catheter ablation using a novel magnetic navigation system: magnetic remote catheter ablation,” Circulation, vol. 109, no. 12, pp. 1472-1475, 2004.
[20] Z. Yang, L. Yang, M. Zhang, C. Zhang, S. C. H. Yu, and L. Zhang, “Ultrasound-guided catheterization using a driller-tipped guidewire with combined magnetic navigation and drilling motion,” IEEE/ASME Transactions on Mechatronics, 2021. https://doi.org/10.1109/TMECH.2021.3121267
[21] Y. Gao, F. Wei, Y. Chao, and L. Yao, “Bioinspired soft microrobots actuated by magnetic field,” Biomedical Microdevices, vol. 23, no. 4, article no. 52, 2021. https://doi.org/10.1007/s10544-021-00590-z
[22] N. Ebrahimi, C. Bi, D. J. Cappelleri, G. Ciuti, A. T. Conn, D. Faivre, et al., “Magnetic actuation methods in bio/soft robotics,” Advanced Functional Materials, vol. 31, no. 11, article no. 2005137, 2021. https://doi.org/10.1002/adfm.202005137
[23] L. Wang, M. Marciello, M. Estevez‐Gay, P. E. D. S. Rodriguez, Y. L. Morato, J. Iglesias-Fernandez, et al, “Enzyme conformation influences the performance of lipase-powered nanomotors,” Angewandte Chemie, vol. 132, no. 47, pp. 21266-21273, 2020.
[24] Y. Wu, Z. Song, G. Deng, K. Jiang, H. Wang, X. Zhang, and H. Han, “Gastric acid powered nanomotors release antibiotics for in vivo treatment of Helicobacter pylori infection,” Small, vol. 17, no. 11, article no. 2006877, 2021. https://doi.org/ 10.1002/smll.202006877
[25] D. Li, C. Liu, Y. Yang, L. Wang, and Y. Shen, “Micro-rocket robot with all-optic actuating and tracking in blood,” Light: Science & Applications, vol. 9, article no. 84, 2020. https://doi.org/10.1038/s41377-020-0323-y
[26] H. Luan and Y. Zhang, “Programmable stimulation and actuation in flexible and stretchable electronics,” Advanced Intelligent Systems, vol. 3, no. 6, article no. 2000228, 2021. https://doi.org/10.1002/aisy.202000228
[27] M. Sitti and D. S. Wiersma, “Pros and cons: magnetic versus optical microrobots,” Advanced Materials, vol. 32, no. 20, article no. 1906766, 2020. https://doi.org/10.1002/adma.201906766
[28] M. T. Thai, P. T. Phan, T. T. Hoang, S. Wong, N. H. Lovell, and T. N. Do, “Advanced intelligent systems for surgical robotics,” Advanced Intelligent Systems, vol. 2, no. 8, article no. 1900138, 2020. https://doi.org/10.1002/aisy.201900138
[29] A. Makkar and J. H. Park, “SecureCPS: cognitive inspired framework for detection of cyber attacks in cyber–physical systems,” Information Processing & Management, vol. 59, no. 3, article no. 102914, 2022. https://doi.org/10.1016/j.ipm.2022.102914
[30] F. Tao, Q. Qi, L. Wang, and A. Y. C. Nee, “Digital twins and cyber–physical systems toward smart manufacturing and industry 4.0: correlation and comparison,” Engineering, vol. 5, no. 4, pp. 653-661, 2019.
[31] S. Kim and Y. Yoon, “ACP model for vehicle monitoring based on CPS,” Human-centric Computing and Information Sciences, vol. 11, article no. 5, 2021. https://doi.org/10.22967/HCIS.2021.11.005
[32] A. E. L. Azzaoui, T. W. Kim, V. Loia, and J. H. Park, “Blockchain-based secure digital twin framework for a smart healthy city,” in Advanced Multimedia and Ubiquitous Engineering. Singapore: Springer, 2021, pp. 107-113.
[33] S. Rathore and J. H. Park, “A blockchain-based deep learning approach for cyber security in next generation industrial cyber-physical systems,” IEEE Transactions on Industrial Informatics, vol. 17, no. 8, pp. 5522-5532, 2021.
[34] D. Ding, Q. Han, Z. Wang, and X. Ge, “A survey on model-based distributed control and filtering for industrial cyber-physical systems,” IEEE Transactions on Industrial Informatics, vol. 15, no. 5, pp. 2483-2499, 2019.
[35] E. A. Lee and S. A. Seshia, Introduction to Embedded Systems: A Cyber-Physical Systems Approach, 2nd ed. Cambridge, MA: MIT Press, 2017.
[36] J. Hwang, J. Y. Kim, and H. Choi, “A review of magnetic actuation systems and magnetically actuated guidewire- and catheter-based microrobots for vascular interventions,” Intelligent Service Robotics, vol. 13, pp. 1-14, 2020. https://doi.org/10.1007/s11370-020-00311-0
[37] Y. Kim, G. A. Parada, S. Liu, and X. Zhao, “Ferromagnetic soft continuum robots,” Science Robotics, vol. 4, no. 33, pp. 1-16, 2019.
[38] Z. J. Liu and J. T. Li, “Accurate prediction of magnetic field and magnetic forces in permanent magnet motors using an analytical solution,” IEEE Transactions on Energy Conversion, vol. 23, no. 3, pp. 717-726, 2008.
[39] D. J. Griffiths, Introduction to Electrodynamics, 4th ed. New York, NY: Cambridge University Press, 2017.
[40] R. Bulirsch, “Numerical calculation of elliptic integrals and elliptic functions. III,” Numerische Mathematik, vol. 13, no. 4, pp. 305-315, 1969.
[41] C. B. Barber, D. P. Dobkin, and H. Huhdanpaa, “The quickhull algorithm for convex hulls,” ACM Transactions on Mathematical Software, vol. 22, no. 4, pp. 469-483, 1996.
[42] R. L. Graham, “An efficient algorithm for determining the convex hull of a finite planar set,” Information Processing Letters, vol. 1, no. 4, pp. 132-133, 1972.
[43] P. T. An, P. T. T. Huyen, and N. T. Le, “A modified Graham’s convex hull algorithm for finding the connected orthogonal convex hull of a finite planar point set,” Applied Mathematics and Computation, vol. 397, article no. 125889, 2021. https://doi.org/10.1016/j.amc.2020.125889
[44] J. Liesen and V. Mehrmann, Linear Algebra. Cham, Switzerland: Springer, 2015.
[45] L. Yang, Z. Q. Zhu, L. Gong, and H. Bin, “PWM switching delay correction method for high-speed brushless DC drives,” IEEE Access, vol. 9, pp. 81717-81727, 2021.
[46] H. Tarzamni, F. P. Esmaeelnia, F. Tahami, M. Fotuhi-Firuzabad, P. Dehghanian, M. Lehtonen, and F. Blaabjerg, “Reliability assessment of conventional isolated PWM DC-DC converters,” IEEE Access, vol. 9, pp. 46191-46200, 2021.
[47] J. Berntsen, T. O. Espelid, and A. Genz, “An adaptive algorithm for the approximate calculation of multiple integrals,” ACM Transactions on Mathematical Software, vol. 17, no. 4, pp. 437-451, 1991.
[48] P. J. Davis and P. Rabinowitz, Methods of Numerical Integration, 2nd ed. San Diego, CA: Academic Press, 1984.
[49] C. Yuksel, “A class of C2 interpolating splines,” ACM Transactions on Graphics, vol. 39, no. 5, article no. 160, 2020. https://doi.org/10.1145/3400301
[50] J. Ratcliff, A. Supikov, S. Alfaro, and R. Azuma, “ThinVR: heterogeneous microlens arrays for compact, 180 degree FOV VR near-eye displays,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 5, pp. 1981-1990, 2020.
[51] M. Magnor, S. Castillo, and A. Sorkine-Hornung, “Real VR,” IEEE Computer Graphics and Applications, vol. 41, no. 4, pp. 27-28, 2021.

Chudong Shan1 , Jianhui Zhao1,* , Wenyuan Zhao2 , Tingbao Zhang2 , Bo Du1 , and Zhiyong Yuan1,*, Magnetic-Actuated Cyber-Physical System for Interventional Surgery, Article number: 12:37 (2022) Cite this article 2 Accesses