Microcapsules of µm order collapse themselves after ultrasound emission. Recently, this technique attracts attention to apply to the gene delivery because virus vector is not necessary. However, it has been limitation to enhance the efficiency of medication because capsules suspension diffuses after the injection, where motion of capsules in blood flow cannot be controlled. To affect behavior of microcapsules, acoustic radiation force was introduced. Then we have detected the local change of microcapsules density by producing acoustic radiation force in the artificial blood vessel. Furthermore, we estimated theoretically condition for active path selection of capsules at the bifurcation in the artificial blood vessel. We observed the difference of density in capsules according to the acoustic radiation force and the produced point. Comparing the experimental results with theoretical equations, the condition for active path selection is calculated from acoustic radiation force and fluid resistance of capsules. It showed a possibility to control capsules direction and to lead to an objective point in blood vessel.
Development of an assistant system using AR technology for echocardiography
We have developed an assitant system for tele-echography between patient and a hospital because of the increase in the number of patients by aging society and recent progress in portable echography. In previous researches three-dimensional position of the ultrasound probe was difficult to specify because a remote doctor observe the patient through a video camera. Therefore we have developed a reproduction system of the probe position using the AR (augmented reality) Toolkit and GUI interface developed by OpenGL. Only an USB camera and two markers for the body surface and the probe are necessary to memorize and transfer three-dimensional position of the probe. As the result of evaluation experiments, guided probe position was satisfied to reproduce the echogram for diagnosis.
Aggregation of fluid microcapsules by producing focused ultrasound in arificial blood vessel
We have developed control method of local density in fluid microcapsules to make use of ultrasound drug delivery system (DDS). It has been difficult to enhance the efficiency of medication because capsules solution diffuses after the injection, where motion of capsules in blood flow cannot be controlled. Then we have noticed that microcapsules are trapped by acoustic standing wave of ultrasound in water. We have applied this method to an artificial blood vessel and observed the variation of capsules density by using the software, which we have developed to detect local change of the brightness variation on echogram. The result indicated that capsules density increases in the upper course of the point where the standing wave is produced. We have also prepared another artificial blood vessel with bifurcation in it. We have recorded echogram of two lower courses after the bifurcation and evaluated the brightness average of the two regions in each course, when the standing wave is produced at a lower course near the bifurcation point. As the result, we confirmed that the density in microcapsules decreases when standing wave is put on the course, where the density increases on the other course. It shows a possibility to control microcapsules direction and to lead to an objective point in blood vessel. [ more info ]
Examination of echocardiography greatly depends on an operator's expertise by concerning the acoustic window, which is discrete area where echocardiogram is obtained clearly. Therefore we have developed spatial recognition software to quantitate the distribution of the acoustic window by extracting the presence of the posterior wall of heart and tracing the probe position three-dimensionally using magnetic sensing device on the probe. In the results of experiment, three-dimensional distribution of the acoustic window was revealed with the outline of the posterior wall to enhance the quantity of diagnosis in echocardiography. This system is useful for the reproduction of echocardiogram and the comparison between multiple operators. [ 
We elucidate viscoelasticity analysis method for robotic echography to creep ultrasound probe on body surface. We have ever experimented and confirmed to be able to obtain echogram of patient. A physician obtained echogram of the heart with sensing contact force of ultrasound probe on body surface. However, the probe is not controlled as the physician's desire on abdomen because the viscosity of soft tissue cannot be ignored. Thus, we have noticed the necessity to investigate character of body surface before attempt to obtain echogram. We propose a method to measure viscoelasticity of the tissue by approaching the probe to the body surface two times and calculate using a Voigt model. 
We have developed a control interface with force sensation for technical simulation of fine adjustment. The system was constructed by link mechanism in combination with wire drive because conventional haptic interfaces are difficult to reproduce pivot operation on soft tissue. The upper point of the handle is controlled by the wire drive, and the lower, by the link mechanism. In this paper we propose a function of variable pivot point to enhance precise pivot operation. Establishing the virtual pivot point, the lower point of the handle is restrained on the line between the upper point and the virtual pivot point. We have confirmed the mobility range and the presented force fulfill the requirement of medical simulation including echography.
We have developed a human interface for medical doctor to control a remote robot by a handle in tele-echography. We enabled bilateral communication of position of ultrasound probe and force restitution using Winsock API and TCP/IP programming. However, there was a problem to confirm position of the probe on a monitor of the remote patient in case of interruption of video stream via network. Thus we added a function to memorize the probe position just before interruption to guarantee the safety for patient using single- and multi- thread on both of server and client. Furthermore, we have visualized the remote probe position with actual handle position as graphics using OpenGL on the same GUI window to evaluate delay time and difference of position between server and client. Using a proxy server in Ehime University, we have verified validity of remote control interface in our laboratory of Tokyo Univ. A&T.
We developed a human interface for medical doctor to control a remote robot by a controller in tele-echography. We have completed signal and image communication between the robot and controller. The remote doctor was able to sense restitution force on patient body surface by introducing bilateral control method. However, position of the ultrasound probe was difficult to understand because all information for the doctor was only on the remote monitor. Therefore we have visualized the both positions of the probe and the actual handle using OpenGL on the same GUI window to evaluate delay time between them. Furthermore, we have introduced Delaunay triangulation method to reconstruct precise shape of body surface considering contact force on it to calculate distance to the body surface for safe control of the robot. We propose an user interface for tele-echography which includes above information.
We have developed an algorithm to find standard cross sections (the long-axis view and the short-axis view) of the heart from successive echograms. We first divided an echogram into small spatial regions and detected the typical motion of the mitral valve by analyzing the brightness variation and correlation coefficient among the regions. We have obtained 95% accuracy in the position of the valve through time series echogram of 25 normal volunteers. The recognized valve was visualized as a mark on the video stream. Furthermore, combining this technique with an optical flow method, we elucidated the region velocity of the wall motion of the left ventricle after centering the valve on echogram. By analyzing symmetry among region velocity, we have confirmed to distinguish between the long- and the short-axis view of heart. This algorism is applicable to instruction software to find standard cross section of the heart as an assistant of echocardiography. We are going to apply to more subjects who have heart disease and to contribute automatic diagnosis in the future. [ 
We have developed an unsymmetrical parallel mechanism with three-dimensional contact force sensation for robotic echography. We improved precision in position, input response and method of installation from our previous robot of twist pantograph mechanism, which satisfied our requirement. By introducing detachable ball joint, installation time to patient is drastically saved. We also considered various body type of patient and three-dimensional contact force of ultrasound probe on body surface. We have experimented and confirmed to be able to obtain echogram of patient by operation of physician. 
Examination of echography greatly depends on expertise and experience of an operator. However, there was no research to assist the way to handle the ultrasound probe. If the cross sections of internal organs are automatically recognized from echogram, position feedback system to operator is possible and that also may enable a guidance system to an inexperienced operator. We have developed an algorithm to recognize a cross section of heart from successive echograms. We first divided an echogram into sub regions and detected the position of mitral valve by analyzing brightness variation and correlation coefficient between sub regions. We confirmed recognition of mitral valve corresponds to the motion of ultrasound probe with reliable accuracy. Furthermore, combining this technique with an optical flow method, we elucidated contract velocity of wall motion in 8 radial regions by centering mitral valve. Finally we recognized a short-axis view of heart by comparing the correlation of contract velocity between specified regions.
We have developed a mechanical interface for virtual echography using a parallel wire drive system. A medical doctor operates the device to control an ultrasound probe which grasped by a remote medical robot on patient abdomen. By using 6 parallel wires, we realized 5 dimensional position comannd by handling a handle in the same way as the usual ultrasound diagnosis, translation, rotatation and touch on the body surface. Also five dimensional force can be presented to controller's hand. It is possible to use for training of echography.
We have developed a method of ultrasound DDS (Drug Delivery System) with microcapsules. Drug targeting is possible to detect microcapsules which contain evaporated drug and collapse them by ultrasound beam emission. Though microcapsules inside body were easily detected by brightness of B-mode echogram, quantitation of amount of released drug was difficult because brightness does not reflect the density of microcapsules. To evaluate collapse of microcapsules, we applied method to measure brightness variation of restricted areas in B-mode echogram. Comparing brightness averages of the areas before and after ultrasound beam emission, we confirmed brightness variation between the two areas. Furthermore, we estimated emitted drug amount by using relation between density of capsules and brightness in B-mode echogram. We propose this method to evaluate effect of dosing in ultrasound DDS.
We have experimented a robotic tele-echography system between a cruising ambulance and hospital. Our robot consists of a twist pantograph mechanism to determine three-dimensional position of ultrasound probe, and gimbals mechanism to determine two-dimensional angle of the probe, respectively. We have reduced the total weight of whole mechanism which compared to the previous version to have it on board. In addition, we have developed a controller to determine posture of the probe as controller's desire and software minding the safety to a patient. As the result, we have succeeded to control the ultrasound probe on remote patient and to obtain clinical echograms.
We have developed a twist pantograph mechanism for robotic tele-echography. We have reduced the total weight of the robot from the previous version. By introducing the inverse kinematics, three-dimensional position and angle of ultrasound probe are able to operated on body surface. Movable area, precision of the position and performance are satisfied with requirement of a diagnosis in echography. We have considered safety by detecting contact force on body surface. We have experimented and confirmed to be able to obtain echogram of remote patient by controlling the robot as physician's desire. 
We have visualized and evaluated collapse phenomena of microcapsules under ultrasound exposure. Brightness of echogram captured microcapsules suspension decreases after ultrasound emission because microcapsules collapse in their resonant frequency. Therefore, comparing successive two echograms, location and degree of microcapsules collapse are indicated. To evaluate degree of microcapsules collapse, we applied method to measure density of capsules from B-mode echogram. This method elucidated relation between density of capsules and brightness of echogram. Variation of density of capsules is calculated throughout digital processing of successive B-mode echograms. We constructed in vitro phantom with fixed microcapsules and confirmed to calculate density of capsules.