2024 Outlook: Focus on Generative AI, Data Quality, and Accessibility
Published: 2024-01-22In 2024, the data and AI industry will keep focusing on generative AI (GenAI) to solve real business problems. This is because there is a growing need for data, especially large language models (LLMs), which require new ways to handle and use the data effectively.
Automated data analysis and activation will become common tools in the industry, helping data teams manage their work like critical product teams. The integration of AI and data will be important, blurring the lines between engineering and data and showing how these fields are connected.
To improve AI products, strategies like retrieval augmented generation (RAG) and fine-tuning techniques will be used. However, making sure the data is good quality will be a big challenge. Strategies for data observability will be necessary to keep the context data for AI products clean and reliable.
For hardware, in-process and in-memory databases will be used to analyze and move datasets. This will make data processing and manipulation more efficient. Lowering cloud costs will also be a priority, and tools for monitoring and optimizing metadata use will help achieve this.
Apache Iceberg, a type of data storage called a data lakehouse table format, will become more popular because it is versatile and scalable.
Even though many companies will make people go back to the office, data and AI teams will still be in high demand. They play a critical role in driving innovation and solving complex business problems.
Researchers from Sahmyook University in South Korea and the Texas A&M Transportation Institute in the US did a study on making subway stations more fair for elderly and disabled people. They used data to see how fair the stations were based on factors like the number of disabled users and the size of the station. The results showed that more investment is needed to make the stations completely fair. Different types of stations need different types of investments to make the vertical transport systems fair. The goal of this strategy is to make the subway accessible for everyone.
The promise of artificial intelligence (AI) has caused a secret war among media moguls, tech entrepreneurs, and financial titans over the data that powers AI systems. Companies are protecting their data because they know that data and AI are important for the future. Lawsuits and alliances are forming as companies fight to keep their data, and legal disputes about data ownership and copyright are becoming more common. The tension between AI models and the data they use will have a big impact on consumers and the future of AI.
New data from a study on a large group of people suggests that using a medicine called glucagon-like peptide-1 receptor agonist (GLP-1 RA) does not increase the risk of pancreatic cancer for up to 7 years. The study looked at over 500,000 adults with type 2 diabetes and found no evidence that GLP-1 RA increased the risk of pancreatic cancer compared to using another medicine called basal insulin. But we need more data and monitoring for longer than 7 years to fully understand the risks. It’s important to know that the study didn’t look at the specific type of GLP-1 RA used.
In conclusion, in 2024, the data and AI industry will focus on generative AI and solving real business problems. AI and data will be integrated, and data quality will be a big challenge. Efforts to make things fair and accessible for everyone, like the strategy for vertical transport facilities, will continue. The fight over data and AI will shape the industry’s future and have a big impact on consumers. Finally, new research shows that using GLP-1 RA is probably not linked to pancreatic cancer, but we need more data to be sure.
https://barrmoses.medium.com/top-10-data-ai-trends-for-2024-7f830196db65?source=home---------1---------------------0787c699_f551_450c_9f8b_f594d66ca0c7-------7