The thin line between AI and data privacy

"Take control of your data” is the theme selected by the National Cybersecurity Alliance for Data Privacy Week this year.

Data privacy remains a core component for both enterprises and customers, especially when it comes to data security. Given the increasing number of data breaches globally, enterprises continue to invest heavily in data protection, ensuring data privacy is intact for customers.

For customers, be it end users of even businesses, ensuring their data is secured and not compromised or used without their permission remains paramount when sharing their information. However, the increasing capabilities of GenAI is starting to change how individuals and enterprise view data privacy.

GenAI capabilities like ChatGPT work best based on the data that’s provided to them. While most of the data used by these models are from public domains, the reality is, users often input their own data to get the best outcome. For example, a user inputs company data to get the tool to come up with a solution. This data is used by the AI tool to generate outputs not just for the user but also train its models to deliver better results.

This is among the main reasons why most enterprises around the world have either banned or limited the use of public AI models like ChatGPT for work. Instead, these companies are now developing their own AI models to improve their productivity and efficiency. Yet, even these models have been highlighted to have weaknesses when it comes to ensuring data privacy and security.

In conjunction with Data Privacy Day, several industry leaders from the tech ecosystem in Asia Pacific shared their views on how AI will influence data privacy in 2025.

David Irecki, Chief Technology Officer for APJ at Boomi believes that data privacy is more critical than ever as organizations invest in AI. He explained that a big part of the solution is making sure organizations only gather data with clear and explicit permission, encrypt personal information, and provide a simple opt-out so customers can decide not to share their sensitive personally identifiable information (PII) data with AI.

“AI transparency also goes a long way. Organizations should clearly explain how their AI model arrives at various decisions and conclusions that make sense to humans. It is key that they have transparent model inputs with AI test policies that generate algorithm results within approved bounds,” said Irecki (pictured below).

Irecki also believes that organizations need to ensure that their models are not trained on data that perpetuates long-standing patterns of discrimination or bias. He added that human oversight remains essential, ensuring these systems comply with regulations and ethical standards.

“Alongside all this, a solid data governance framework is crucial to preserve trust and maintain consistent, high-quality information and protect sensitive data. With responsible practices and clear accountability, AI can thrive without sacrificing privacy,” he said.

A business imperative

Meanwhile, Adhil Badat, Chief Operating Officer for Asia-Pacific Japan at Rackspace Technology believes that data privacy has evolved from a technology concern to an essential business imperative, especially in an era when consumers are demanding stronger accountability from brands and companies in 2025. For Badat, safeguarding customer data and building trust are critical for organizations not just to survive but to grow their business amid various economic and specialized disruptions.

“The good news is technological advancements have significantly enhanced data privacy efforts. Automated solutions, especially with the implementation of artificial intelligence and machine learning, can now monitor and enforce compliance standards in real time, reducing human error. Integrating AI into these solutions has further augmented organizational IT capabilities, increasing the effectiveness of fundamental data protection controls. Innovations in data security, such as encryption and tokenization, bolster resilience against breaches by making it difficult for unauthorized users to access or misuse data,” explained Badat (pictured below).

Additionally, Badat also mentioned that the evolution of authentication methods, including password-less options and biometric indicators, provides robust defenses against unauthorized access.

“An often overlooked yet essential aspect of data privacy is employee training. Equipping staff with knowledge of best practices in data security remains a vital component of an organization's overall strategy to protect sensitive information,” he added.

For Arun Kumar, regional director for APAC, at ManageEngine, data privacy requires a comprehensive approach that integrates people, processes, and technology frameworks.

"Solutions like security information and event management (SIEM) leverage AI and automation to proactively identify, manage, and neutralize potential threats. These tools provide organizations with real-time alerts, enhancing their ability to respond swiftly to security incidents. However, technology alone is not enough. Data privacy also involves educating employees and fostering a culture of shared responsibility where everyone adheres to robust data protection policies and practices,” said Kumar.

Mobile data privacy

When it comes to data privacy for mobile data, Jan Sysmans, Mobile App Security Evangelist, Appdome stated that mobile apps are central to consumer and business interactions, making robust security a necessity for developers.

“Evolving threats like AI-driven attacks and account takeovers exploit app vulnerabilities, compromising user accounts, data, and brand trust. To combat these risks, mobile businesses must adopt comprehensive, lifecycle-focused security measures. At the same time, protecting user data is about more than compliance—it’s about fostering trust and loyalty throughout the customer journey. With Appdome, mobile businesses can secure sensitive data and mobile accounts at every interaction, delivering peace of mind for both businesses and users,” said Sysmans (pictured below).

Securing digital identities

Given the increasing capabilities of GenAI, digital identities are becoming a normality today, not just for employees but for any consumer. Digital identities are built and based off personal data. Hence, ensuring data privacy for digital identities is even more essential.

According to Jasie Fon, Regional Vice President of Asia at Ping Identity, it is essential to recognize that data privacy isn't just a compliance requirement—it's a matter of trust and transparency.

“According to our survey last year, most Singapore consumers (86%) do not fully trust the organizations that manage their identity data. Doing online shopping (72%) made consumers feel most vulnerable to identity theft, followed by accessing or using online banking (69%) and using social media (68%),” said Fon.

Fon explained that Digital identity is the front door to any digital experience. As such, organizations need to ensure that businesses are investing in the latest technologies such as customer identity and access management (CIAM) and decentralized identities (DCI) to secure consumers’ identities and prevent fraud.

“Organizations are also responsible for ensuring customers understand how data is collected and are given a clear opt-in or opt-out option to feel secure and respected. This transparency goes a long way toward building brand loyalty and a positive customer experience,” added Fon.

Echoing Fon’s views is Lim Teck Wee, Area Vice President for ASEAN at CyberArk. Lim pointed out that several countries in the region, including Singapore and Indonesia, have strengthened data protection laws in the past year, reflecting a growing recognition of the importance of safeguarding personal information.

“The ever-expanding volumes of data, rapid advancements in technologies like AI, and increasingly sophisticated threat actors demand unwavering focus and action. Identity security is a critical pillar of data privacy. Deploying robust workforce identity management solutions and protecting user credentials are key steps in preventing breaches. Prioritizing identity security will not only ensure compliance with local data protection laws; it will also ensure that organizations are mitigating risks and protecting customer trust,” concluded Lim.