Privacy-Preserving Visual Content Tagging using Graph Transformer Networks
With the rapid growth of Internet media, content tagging has become an important topic with many multimedia understanding applications, including efficient organisation and search. Nevertheless, existing visual tagging approaches are susceptible to inherent privacy risks in which private information may be exposed unintentionally. The use of anonymisation and privacy-protection methods is desirable, but with the expense of task performance. Therefore, this paper proposes an end-to-end framework (SGTN) using Graph Transformer and Convolutional Networks to significantly improve classification and privacy preservation of visual data. Especially, we employ several mechanisms such as differential privacy based graph construction and noise-induced graph transformation to protect the privacy of knowledge graphs. Our approach unveils new state-of-the-art on MS-COCO dataset in various semi-supervised settings. In addition, we showcase a real experiment in the education domain to address the automation of sensitive document tagging. Experimental results show that our approach achieves an excellent balance of model accuracy and privacy preservation on both public and private datasets.
Code (1)
Tasks
graph constructionKnowledge GraphsPrivacy PreservingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Privacy-Preserving Visual Content Tagging using Graph Transformer Networks
With the rapid growth of Internet media, content tagging has become an important topic with many multimedia understanding applications, including efficient organisation and search. Nevertheless, existing visual tagging a…
graph constructionKnowledge GraphsPrivacy PreservingAn Investigation towards Differentially Private Sequence Tagging in a Federated Framework
To build machine learning-based applications for sensitive domains like medical, legal, etc. where the digitized text contains private information, anonymization of text is required for preserving privacy. Sequence taggi…
Federated Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Beyond Pixels: Semantic-aware Typographic Attack for Geo-Privacy Protection
Large Visual Language Models (LVLMs) now pose a serious yet overlooked privacy threat, as they can infer a social media user's geolocation directly from shared images, leading to unintended privacy leakage. While adversa…
On Content-Based Recommendation and User Privacy in Social-Tagging Systems
Recommendation systems and content filtering approaches based on annotations and ratings, essentially rely on users expressing their preferences and interests through their actions, in order to provide personalised conte…
Recommendation SystemsTAGCSA-Graphs: A Privacy-Preserving Structural Dataset for Child Sexual Abuse Research
Child Sexual Abuse Imagery (CSAI) classification is an important yet challenging problem for computer vision research due to the strict legal and ethical restrictions that prevent the public sharing of CSAI datasets. Thi…