Development of Fine-grained Pill Identification Algorithm using Deep Convolutional Network JBI'17
Introduction
Oral pills, including tablets and capsules, are one of the most popular pharmaceutical dosage forms available. Compared to other dosage forms, such as liquid and injections, oral pills are very stable and are easy to be administered. However, it is not uncommon for pills to be misidentified, be it within the healthcare institutes or after the pills were dispensed to the patients. Our objective is to develop groundwork for automatic pill identification and verification using Deep Convolutional Network (DCN) that surpasses the existing methods.
Highlights
- A DCN model was developed using pill images captured with mobile phones under unconstraint environments. The performance of the DCN model was compared to two baseline methods of hand-crafted features.
- The DCN model outperforms the baseline methods. The mean accuracy rate of DCN at Top-1 return was 95.35%, whereas the mean accuracy rates of the two baseline methods were 89.00% and 70.65%, respectively. The mean accuracy rates of DCN for Top-5 and Top-10 returns, i.e., 98.75% and 99.55%, were also consistently higher than those of the baseline methods.
- The images used in this study were captured at various angles and under different level of illumination. DCN model achieved high accuracy despite the suboptimal image quality
- The superior performance of DCN underscores the potential of Deep Learning model in the application of pill identification and verification.
Citation
-
Development of Fine-grained Pill Identification Algorithm using Deep Convolutional Network
Y. F. Wong, H. T. Ng, K. Y. Leung, K. Y. Chan, S. Y. Chan, C. C. Loy
Journal of Biomedical Informatics, 2017 (JBI)
DOI
Images
Pills were categorized in accordance to their dosage form, shape, presence or absence of imprint and color. 81.5% of the pills were tablet and 83% were imprinted with symbols or letters.
Example of images taken at various angles, from different distances, and under different illumination condition.
(A) The general pipeline for pill identification: (Step-1) Perspective correction, (step-2) pill detection, (step-3) pill identification using a trained deep model. (B) From left to right: Pill image after (i) perspective correction, (ii) superpixels, (iii) first stage of saliency detection, and (iv) second stage of saliency detection. The foreground segment was then obtained by performing adaptive thresholding on the second-stage output of saliency detection.
Comparison of mean accuracy rates (%) and standard deviations across five repeated random sub-sampling (mean ± s.d)
Two-dimensional embedding of pill patterns obtained using multi- dimensional scaling. Each point corresponds to representation of a pill obtained through (A) manually designed features and (B) hidden features extracted from the fully connected layer of deep convolutional network. Every point is encoded by color based on its associated class. The positions of six distinct pills are also shown in (B).
(A) Failure cases of baseline that exploits manually designed features with random forest as classifier. The baseline fails on all these while Deep Convolutional Network (DCN) succeeds in all except for the last one, where pill Class 20 was mistaken as Class 21 (highlighted in red borders). (B) Pill queries that DCN misclassify were mostly those that bear minimal identification features, namely of standard shapes and without any imprint. The left image of each image pair is the query, while the right one is the search result returned by the baseline or the DCN.
Supplementary Materials
- To obtain the dataset, please send an email to the corresponding author, Yuen Fei Wong (yfwong at um dot edu dot my).