Privacy-Preserving Image Captioning with Deep Learning and Double Random Phase Encoding

Abstract

Cloud storage has become eminent, with an increasing amount of data being produced daily; this has led to substantial concerns related to privacy and unauthorized access. To secure privacy, users can protect their private data by uploading encrypted data to the cloud. Data encryption allows computations to be performed on encrypted data without the data being decrypted in the cloud, which requires enormous computation resources and prevents unauthorized access to private data. Data analysis such as classification, and image query and retrieval can preserve data privacy if the analysis is performed using encrypted data. This paper proposes an image-captioning method that generates captions over encrypted images using an encoder–decoder framework with attention and a double random phase encoding (DRPE) encryption scheme. The images are encrypted with DRPE to protect them and then fed to an encoder that adopts the ResNet architectures to generate a fixed-length vector of representations or features. The decoder is designed with long short-term memory to process the features and embeddings to generate descriptive captions for the images. We evaluate the predicted captions with BLEU, METEOR, ROUGE, and CIDEr metrics. The experimental results demonstrate the feasibility of our privacy-preserving image captioning on the popular benchmark Flickr8k dataset. © 2022 by the authors.