The maritime sector is an integral component of the global economy, and its necessity is emphasized more than ever because trade and relations between nations will halt without it. It serves as the hub for global cargo transportation, and the growth of a nation’s economy depends heavily on the effectiveness and efficiency of its marine transport system.
It is important to recognize that the maritime industry plays a crucial role in modern countries’ social and economic advancement and has significantly raised living standards worldwide.
While a large portion of the sector works with transporting commodities, products, and people by sea—including anything from container ships and oil tankers to small boats and passenger ferries— this is not the only purpose.
The maritime industry also includes the construction, repair, and maintenance of ships, the development of port facilities, and marine engineering. Around the world, the maritime sector is also a significant employer of labor.
The International Maritime Organization (IMO) estimates that more than 1.8 million sailors work worldwide. This excludes the estimated 20+ million workers, such as shore-based personnel, workers in the port facilities, constructors, and marine engineers, who serve or engage with the maritime industry in some way.
The fact that water transportation is one of the most environmentally friendly modes of transportation is also a positive.
With the development of science and technology, we have witnessed the metamorphosis of all major industries, and the marine sector has had its fair share of change. Now, Big Data in the maritime industry is the new revolution.
An enormous amount of data is produced in an industry like the maritime industry, which manages many people and cargo. And data is everything in the twenty-first century. Data enables commercial decision-makers to base their choices on facts, statistical data, and trends.
In its most basic sense, Big Data refers to the enormous quantities of organized and unorganized data that give businesses and sectors evidence-based perspective into their present and future customer and market needs. Big data analytics are used to interpret large amounts of data, using a blend of Artificial Intelligence and Machine Learning-driven tools, Algorithms, and processing systems.
Industries like maritime have conventionally utilized enterprise resource planning (ERP) and other dispersed storage systems to utilize data. But, with the development of Big Data analytics, there is no better supply chain visibility.
Big Data offers the crucial clarity required in the complex supply chain landscape of the shipping industry. It enables industries to identify all of their problems, which may not be apparent if you only look at the fundamental data at a primary level.
Understanding how Big Data is reshaping an industry that fuels global trade is crucial for anybody interested in maritime. Here are some of the ways that Big Data is impacting the maritime industry:
Better Decision Making
Companies have more control over their shipping strategy and boost their overall key performances because every bit of information would be placed under a single cloud with Big Data, making the calculations much more straightforward. This is due to the availability of pertinent information at the appropriate time.
Making organized decisions with a strong foundation of available data is made easier with the relevant information. Every choice is data-driven, which enables an efficient industry. Nothing is left to chance or irrational intuition.
Improved Visibility
Visibility along transportation routes is becoming more and more necessary in the sector. Shipping managers are changing how they oversee the transport lines to ensure improved output.
From the origin to the destination, authentic freight analytics are required. With the constant real-time data stream, managers may quickly discover shortcomings and inefficiencies.
As a result of the data systems’ thorough and more frequent updates, all parties affected are automatically updated and informed. By offering an automated and proactive method that directly utilizes Big Data analytics, the supply chain may be made entirely immune to disruptions.
Enhanced Logistics:
Using data science in logistics can assist marine organizations in more effectively maximizing operations. This covers many concerns, such as the best delivery routes, better fuel management techniques, the best times of day to travel, and more precise supply and demand forecasting.
Applying data science to improve logistics can also assist businesses in using instantly supplied insights to make modifications along the route, as different dynamic elements like customer demand or gas costs can be acted on more quickly.
Using Big Data science and analytics is more efficient, less expensive, and faster to perform any required calculations or recalculations during operations.
This is a massively important activity for credit card firms, who can determine when spending slips outside a regular pattern for that consumer. The payment can then be flagged to prevent a possible case of fraud.
Online businesses can be approached in the same way. The shop can set up alerts if it notices multiple payment methods from the same IP address. In maritime, a string of bookings that deviates from a pattern may also be considered fraudulent activity.
Marketing and Advertising:
Big Data is advantageous even at the level of retailers since it can be used to identify trends in personalized marketing, which retailers can then utilize to create more individualized experiences by using targeted advertising.
The business can segment its audience to deliver customers more customized products and discount offers. Age, Sex, Location, and other demographic and socioeconomic data can all be used for segmentation.
This personalization can also encompass style and size of preferences and influence customers’ purchasing decisions.
Design and Tracking:
Ship design also makes use of big data. It may involve examining sensors on ships that are currently at sea. It may also involve looking at storage data, engine performance, cargo protection, and smoother operation.
While sensors, barcodes, and Radio frequency identification (RFID) Tags can be used in various ways to track and monitor shipments, integrating them with data on the weather and traffic can assist with making projections about arrival times, especially if there are delays.
Wrapping Up
Over the last several years, most other sectors have been utilizing Big Data’s power to speed up innovation and efficiency.
The marine sector is now also adopting digitization and appreciating the value of data and technology in making it a more vibrant, competitive sector of the 21st century.
AI has been a highly useful technology for marketers over the past few years. However, new advances in AI continue to make drive new changes in the marketing profession. As a result, analysts anticipate that the market size for AI technology in the marketing sector will be worth nearly $108 billion by 2028.
Marketers Utilize AI Technology to Leverage 3D Rendering Capabilities
Modern 3D rendering allows designers to turn their technical drawings into visual representations of their products. Since the days of Toy Story 1, we have seen the CGI (computer-generated images) industry grow and become more sophisticated. Nowadays, anybody can learn how to create 3D-modeled and rendered images. In fact, AI technology has made it easier than ever! These days, you can create a design from the comfort of your home PC, send it off to be rendered, and then see it here and all without spending a fortune or taking expensive college courses. It is no surprise that even furniture companies are getting in on the action.
TechCrunch has a great post detailing the newest AI model for 3D rendering models. This model was created by OpenAI and it was built as an open-source platform created with Point-E. Point-E is an AI system that creates 3D objects from text prompts. Point-E takes one to two minutes to generate models on a single Nvidia V100 GPU, and produces point clouds rather than traditional 3D shapes. However, it has not been able to capture finer details such as textures and shape. Nonetheless, this system is more efficient than previous approaches.
OpenAI researchers say Point-E can quickly produce colored point clouds matching text prompts, after the researchers trained their cutting-edge models on a dataset of several million 3D objects and those objects’ metadata. It doesn’t always perform perfectly; sometimes, it misunderstands the image based on the text-to-image model, producing an irrelevant shape compared to what was asked for. Still, OpenAI claims its method is orders of magnitude faster than prior state-of-the-art.
AI Allows You to See Photo-Realistic Designs Before They Are Manufactured
Let’s say you have a series of furniture designs and you want to do a little market research prior to actually having them created. This is one of the biggest ways that AI has changed the furniture design process.
Thanks to the use of AI-driven 3D modeling, designs can be created and rendered as images or even as objects that can be manipulated via something as simple as a web browser. People may see and examine the various pieces of furniture, give their opinions and even make orders in advance. This works well for large businesses that want to conduct market research and smaller companies that wish to take in some custom orders.
Offer People Some Alternatives To Your Designs
You have probably seen those websites where they offer something like a sweater in one color, but you can press one of the many color buttons and it shows you the exact same sweater but in a different color. You can do similar things with rendered images and models of furniture by using advanced AI tools. The differences can be added and removed as easily as you toggle a sweater’s color online. For example, do you want brass handles, do you want the darker oak color, do you want shorter feet? Toggle the buttons and the 3D-rendered image changes to meet your demand.
Being As Creative As They Like
Modern 3D modeling and rendering allow people to be creative because they can have crazy ideas in their heads and actually see a photo-realistic model of their ideas without ever having to spend money manufacturing it. Oddly, when a crazy idea turns out to look really good or works really well, then the 3D-rendered image can be used to sell the idea to investors (or bosses), who will then give the go-ahead to build the furniture.
Remember the popularized throne in the TV show “Game of Thrones.” Imagine some designer coming up to you and saying they want to make a throne out of swords. You would probably dismiss them outright. But imagine they showed you a 3D-rendered model of the throne chair. It would change your whole perspective.
Show Furniture in Situ
Companies can market their furniture with apps and even with dynamic websites. They can have people take photos of their rooms, or upload images of their rooms, and then allow them to drag and drop furniture into the room.
For example, let’s say you have just created a new table. With the power of 3D rendering, people could place that table into photos of their own living room. People could try out several different types of table until they find the one they like. It is a little like trying before you buy.
Demonstrate Furniture With AR and VR
Now that Augmented Reality and Virtual Reality are a thing, we will soon see more and more furniture stores offering the same things as mentioned above, but in AR or VR. People will be able to walk through their living room with AR or VR and pick and choose which pieces of furniture they would like. They could create whole rooms in virtual reality and then order those pieces from furniture merchants and manufacturers.
AI Helps Marketers Make the Most of 3D Rendering Technology
AI technology has led to some major changes in the marketing sector in recent years. One of the biggest changes has been the evolution of 3D rendering. A growing number of marketers are using AI to improve their product designs and create better marketing materials as a result.
Editor’s note: This blog is part of a series called Meet the Google Cloud Data Champions, a series celebrating the people behind data- and AI-driven transformations. Each blog features a champion’s career journey, lessons learned, advice they would give other leaders, and more. This story features Jan Riehle, Principal at Brazilian investment firm Rising Venture and founder and CEO of a Brazilian company that runs a beauty e-commerce platform, B4A.
Tell us about yourself. Where did you grow up? What did your journey into tech look like?
I grew up in Karlsruhe, a town in southern Germany. Directly after high school, in the early 2000s, I opened my first tech company, an agency creating web technology for small and medium-sized businesses.
Several years later, I relocated to Switzerland, Singapore, and France, and I took roles in several tech companies while also collecting experiences in M&A and private equity. After pursuing an MBA at INSEAD, I relocated to Brazil in 2011, where I co-founded and ran various technology ventures between 2011 and 2015.
In 2017, I started a search fund that acquired two businesses in the “beauty-tech” space in Sao Paulo, Brazil’s commercial capital. These two businesses were the beginning of what today is B4A (“Beauty for all”), a platform that creates an ecosystem, connecting and mutually benefiting consumers, beauty brands, and digital beauty influencers.
What’s the coolest thing you and/or your team have accomplished by leveraging our Data Cloud solutions?
Our main objective is to provide value for our ecosystem participants, which are consumers, beauty brands and digital influencers. All of our technology efforts and success are made to serve that purpose and create value for these three groups.
The coolest thing we achieved by using Google’s Data Cloud solutions was to extremely shorten load times for our consumer-facing ecommerce platform, B4A Commerce Connect. You can see it in action at www.glambox.com.br or www.mensmarket.com.br. The performance gains are visible when you load heavy collections (like the “Para Ele” Collection on the Glambox website, for example). For such large collections, we were able to reduce load times by about 90% by implementing AlloyDB for PostgreSQL.
Our platform combines data about customer characteristics with machine-learning algorithms so that a user of our website only sees products that make sense for their individual profile. This raises a challenge because every load in our ecommerce platform requires more computing power than in a standard ecommerce platform, where such optimizations are not needed. Therefore, using the right tools and optimizing performance becomes absolutely crucial to provide smooth, fluid performance and a solid user experience. The user experience benefited dramatically from implementing AlloyDB. You can learn more about our journey in this blog.
Technology is one part of data-driven transformation. People and processes are others. How were you able to bring the three together? Were there adoption challenges within the organization, and if so, how did you overcome them?
B4A is a beauty company with technology in its DNA (we also call it a “beauty-tech”). We always strive to use the best technologies for the benefit of our ecosystem participants. Implementing solutions from Google Cloud was very beneficial to our processes and did not result in any additional challenges, nor resistance from the team. We actually had the opposite reaction, to be honest: infrastructure requirements and maintenance efforts decreased by more than 50%, which our IT Operations team very much welcomed. At the same time, and as I mentioned before, our customers also benefited from integrating Google Cloud, and specifically AlloyDB, creating a win-win situation for the organization.
What advice would you give people who want to start data initiatives in their company?
The starting point is the most fundamental moment. It will determine the success of your implementation. After all, a small difference in your steering angle at the start will make a big impact at the end. It’s not easy to set a course when you don’t know where you want to go. So, even if you are far away, you need to have a clear vision about where you want to go and a roadmap of how to get there.
With that in mind, I recommend preparing a data organization framework that will be able to support your plan. Even if you start small, you’ll need to set aside time to document, and cross-functionally review what you’ve envisioned.
Before you jump into action and develop a technology project, try to have a clear blueprint about your data structure. Map out all the data you need to track and try to predict what you will need in the future. The better you plan in the beginning, the better your end result will be.
What’s an important lesson you learned along the way to becoming more data-driven? Were there challenges you had to overcome?
I think in terms of data, we are different from most organizations. Our relationship with data has always been pronounced, and even our organizational structures are designed to use data in the best possible way, with data-focused squads accompanying many organizational processes. We already have a five-year track-record of extensive data usage at B4A. In the end, we need data for our main products to work, and we also sell it in an aggregate form to beauty brands.
The first important learning I already provided in my previous answer: above all, it is important to define data structures and oversee company-wide processes well before starting to implement an actual database. In this step, it is extremely important that business and tech teams work hand-in-hand. The business side needs to have a certain degree of technical thinking in order to make this integration work in a productive way. Overall, the better you plan in the beginning, the better your end result will be.
The second learning is more subtle and it’s something I only realized recently, after years of using data for everything at B4A. The learning is that looking at the data’s current trajectory, rather than envisioning its potential trajectory, can actually limit an organization.
Data-driven organizations can train themselves to assume past data will evolve in a linear fashion. The problem with this approach, when you are in technology, is that you often work on potentially disruptive products. Instead of just looking at problems in a linear fashion, you should also think of the potential exponential curves that may develop once your features or products achieve a sufficient degree of product-market fit.
This different behavior is often not considered when making projections using classical regressions or linear projections on top of data. Therefore, I sometimes provoke the team to look at the data from a different angle—an angle of where we want to go and how strong the disruptive effect of a new feature potentially could be in the market. In the end, the organization needs to find a way to combine both approaches and balance one with the other.
Another best practice is to avoid letting the organization become obsessed with certain key metrics, or KPIs. When looking at metrics, never forget that they often reflect simplifications of reality. The context in the real world can be more complex, integrating many more variables that should be considered to get a full picture of a situation. Applying common sense should always be more important than trying to judge a situation only using one or several metrics.
Thinking ahead 5-10 years, what possibilities with data and/or AI are you most excited about?
I think we are only at the very beginning of AI having an impact on productivity. Looking at five or even ten-year periods, it is difficult to predict the amount of disruption that will happen from AI. It’s already starting with generative AI, and I think over time, the entire Software-as-a-Service industry will end up being disrupted by more “Model-as-a Service” oriented companies. Instead of using a software product, you may be able to just ask the model to provide whatever you need at a specific moment in a very customized way.
At B4A, we always strive to be at the top of new developments. We consider how we can implement them for not only the interest of our ecosystem participants, but also the interest of our company’s efficiency. Again, looking at the next ten years, I think the disruption from data and AI will be immense, larger than anything we have seen over the last 50 years.
Want to learn more about the latest innovations in Google’s Data Cloud across databases, data analytics, business intelligence, and AI? Join us at the Google Data Cloud & AI Summit to gain expert insights and data strategies to drive transformation in your organization.
Fully unleashing the power of data is an integral part of many companies’ goals of digitalization. At SHAREit Group, we heavily rely on data analytics to continue optimizing our services and operations. Over the past years, our mobile app products, notably the file sharing platform SHAREit, which aims to make content equally accessible by everyone, has quickly gained popularity and reached more than 2.4 billion users around the world. We could never achieve this without the insights we gained from our business data for product improvement and development.
SHAREit Group has adopted a multicloud strategy to deploy our products and run data analytics since our early days because we want to avoid provider lock-in, take advantage of the best-of-breed technologies, and avoid potential system failures in the event that one of our multiple cloud providers encounters technical issues. As an example, we use Google Cloud, because its infrastructure tools like Google Kubernetes Engine (GKE) and Spot VMs help us further lower our computing costs, while the combination of BigQuery and Firebase for data processing speeds up our data-driven decision-making.
To easily build a unified user experience across all our different cloud platforms, we rely on different open source tools. But using multiple public clouds and open source software inevitably complicates the ways we gather and manage our ever-increasing business data. That’s why we need a powerful big data platform that supports highly efficient data analytics, engineering and governance across different cloud platforms and data processing tools.
Current challenges for big data platforms
As the quantity of data continues to increase and the applications of data diversifies, the technology for big data platforms has also evolved. However, many obstacles like poor data quality and long data pipelines are still preventing companies from getting the most value out of the data they have. On our journey to build an enterprise-level, multicloud big data platform, we’ve encountered the following challenges:
Long data cycle: A centralized data team can help process data across different company systems in a more organized way, but this centralization also prolongs data cycles. Data specialists might not be familiar with how different domain teams use data, and it requires a lot of back-and-forth communication before raw data are transformed into useful information, which results in low efficiency in decision-making.
Data silos: We use several online analytical processing (OLAP) engines for our different systems. Each OLAP tool has its own metadata, which brings about data silos that prevent cross-database queries and management.
Steep learning curve: To utilize data in different databases, users need to have a good command of various SQL languages, which translates into a steep learning curve. On top of that, finding the most ideal pair of SQLs and query engines to process data workloads can be challenging.
High management costs: Enhancing the cost-effectiveness of our cloud infrastructure is one of the main reasons why we adopted a multicloud architecture. However, many cloud-based big data platforms lack a mechanism of using cloud resources cost-efficiently. The management cost could have been significantly lower if we were able to avoid waste of CPUs and memory.
Low transparency: The information about data assets and costs across different databases is often scattered on big data platforms, which makes it challenging to realize efficient data governance and cost management. We need a one-stop solution to fully eliminate excessive data and computing resources.
DataCake: A highly efficient, automated one-stop big data platform supported by Google Cloud
To overcome the above-mentioned challenges, SHAREit Group in 2021 started using DataCake, to support all our data-driven businesses. DataCake facilitates the implementation of the data mesh architecture, a domain-oriented, self-serve data platform design that enables domain teams to conduct cross-domain data analysis independently. By supporting highly automated, no-code data analytics and development, DataCake lowers the bar for any user who wants to make use of data.
In addition, DataCake is built on multicloud IaaS, which allows us to flexibly leverage leading cloud tools like GKE, the most scalable and automated Kubernetes, and Spot VMs to realize the most cost-effective use of cloud resources. DataCake also supports all types of open source query engines and business intelligence tools, facilitating our wide adoption of open source software.
Key benefits of using DataCake include:
Highly efficient data collaboration: While giving full data sovereignty to each domain team, DataCake offers several features to facilitate data collaboration. First, it provides standard APIs that allow different domain teams to easily share data by one click. Secondly, LakeCat, a unified metastore service in DataCake, gathers all metadata in one place to simplify management and enables quick metadata queries. Thirdly, DataCake supports queries across 18 commonly used data sources, which enhances the efficiency of data exploration. According to the TPC-DS benchmark, DataCake delivers 4.5X higher performance than open source big data solutions.
Lower infrastructure costs: Leveraging multicloud IaaS means that DataCake gives its users full flexibility of choosing the cloud infrastructure tools that are most cost-effective and meet their needs the best. DataCake’s Autoscaler feature supports different virtual machine (VM) instance combinations and can help maintain a high usage rate of each instance. By optimizing the ways we use cloud infrastructure, DataCake has helped SHAREit group lower data computing costs by 50%.
Less query failure: Choosing the most suitable query engine for workloads using different SQLs can be a headache for data teams that leverage multiple query engines. At SHAREit Group, we employ not only open source data processing tools like Apache Spark and Apache Flink, but also cloud software including BigQuery. DataCake’s AI model, which is trained with SQL fragments, is able to select the most ideal query engine for a workload based on its SQL features. Overall, DataCake reduces query failure caused by unfit engines by more than 70%.
Simplified data analytics and engineering: DataCake makes data analytics and engineering feasible for everyone by adopting serverless PaaS and streamlining SQL use. With serverless PaaS, users can focus on data-related workloads without worrying about cluster management and resource scaling. At the same time, DataCake provides all types of development templates and a smart engine to automate SQL deployment, which allows users to complete the whole data engineering process without using any code.
Comprehensive data governance: On DataCake, users can see all their data assets and billing details in one place, which makes it easy to manage data catalogs and costs. With this high level of transparency, SHAREit Group has successfully saved 40% of storage costs.
How Google Cloud supports DataCake
In early 2022, SHAREit Group started incorporating Google Cloud into our multicloud architecture that underlies DataCake. We made this decision not only because we wanted to increase the diversity of our cloud infrastructure, but also because Google Cloud offers opportunities to maximize the benefits of using DataCake by further lowering costs and facilitating data analytics. Leveraging Google Cloud to support DataCake has given us the following advantages:
Lower computing costs: Spot VMs of Google Cloud are one of the VMs with the lowest price-performance ratio on the market, and DataCake’s Autoscaler feature can make the most out of this advantage by predicting the health status of Spot VMs to reduce the probability of them being recycled and disrupting computing. On top of that, DataCake built an optimized offline computing mechanism to avoid redoing computing through persistent volume claims. All in all, we’ve reduced the execution time of the same computing task by 20% to 40%, and realized 30% to 50% lower computing costs.
Lower cluster management costs: Google Cloud is highly compatible with open source tools and can help realize cost-effective open source cluster management. With the autoscaling feature of GKE, our clusters of Apache Spark, Apache Flink and Trino are automatically scaled up or down according to current needs, which helps us save 40% of cluster management costs.
More cost-effective queries: We use BigQuery as a part of PaaS supporting DataCake. Compared to other cloud warehouse tools, BigQuery offers more flexible pricing schemes that allow us to greatly reduce our data processing costs. Additionally, the query saving and sharing feature of BigQuery also facilitates the collaboration between different departments, while its capability to generate several terabytes of data in only a few seconds accelerates our data processing speed.
By merging Google Cloud and DataCake, we’re able to take advantage of the powerful infrastructure of Google Cloud to fully benefit from DataCake’s features. Now, we can conduct data analytics and engineering in the most cost-effective way possible and have more resources to invest in product development and innovation.
Continue democratizing data
SHAREit Group is happy to be part of the journey for data democratization and automation. With the help of Funtech and Google Cloud, SHAREit will continue to innovate with better data analytics capabilities and we’ll keep finding new ways to strengthen the edges of our big data platform on DataCake by leveraging the cutting-edge technologies of public cloud platforms like Google Cloud.
Special thanks to Quan Ding, Data Analyst from SHAREit, for contributing to this post.
Businesses rely on an inflow of documents to drive processes and make decisions. As documents flow into a business, many are not classified by type, which makes it difficult for businesses to manage at scale.
At Google Cloud, we’re committed to solving these challenges with continued investment in our state-of-the-art machine learning product for document processing and insights: Document AI Workbench, which helps users quickly build models with world-class accuracy trained for their specific use cases. In February 2023, we launched the Custom Document Extractor (CDE) in GAto help users extract structured data from documents in production use cases. Today, we’re announcing the newest model type to help users automate document processing, Custom Document Classifier (CDC). With CDC, users can train highly accurate machine learning models to automatically classify document types.
CDC provides tangible business value to customers. For example, businesses can validate if users submit the right documents within an application, lowering review time and cost. In addition, accurate classification enables businesses to better automate downstream processes. This includes selecting the proper storage, analysis, or processing steps.
In this blog post, we’ll give an overview of the Custom Document Classifier and ways customers are already benefiting from it.
Benefits of classification models with Document AI Workbench
Our customers use Document AI Workbench to ultimately save time and money, building models with state of the art accuracy in a fraction of the time that traditional development methods require. Thus, CDC helps businesses achieve higher automation rates to scale processes while lowering costs.
Chris Jangareddy, managing director for Artificial Intelligence & Data at Deloitte Consulting LLP said, “Google Cloud Document AI is a leading document processing solution packed with rich features like multi-step classify and text extraction to automated sorting, classification, extraction, and quality assurance. By combining Document AI with Workbench, Google Cloud has created a forward-thinking and powerful AI platform for intelligent document processing that will allow for process transformation at an enterprise scale with predictable outcomes that can benefit businesses.”
Rajnish Palande, VP, Google Business Unit for BFSI, TCS said, “Document AI Workbench leverages artificial intelligence to manage and glean insights from unstructured data. Workbench brings together the power of classification, auto-annotation, page number identification, and multi-language support to help organizations rapidly deliver enhanced accuracy, improved operational efficiency, higher confidence in the information extract, and increased return on investment.”
Sean Earley, VP of Delivery Services of Zencore said, “Document AI Workbench allows us to develop highly accurate document parsing models in a matter of days. Our customers have automated tasks that formerly required significant human labor. For example, using Document AI Workbench, a team of two trained a model to split, classify, and extract data from 15 document types to automate Home Mortgage Disclosure Act reporting. The mean trained model accuracy was 94%, drastically reducing the operational cost of our customer’s compliance reporting procedures.”
How to use Custom Document Classifier
Users can leverage a simple interface in the Google Cloud Consoleto prepare training data, create and evaluate models, and deploy a model into production, at which point it can be called to classify document types. You can follow the documentation for instructions on how to create, train, evaluate, deploy, and run predictions with models.
Import and prepare training data
To get started, users import and label documents to train an ML model. Users can label documents in bulk at import to build the training and test datasets needed to build a model accurate enough for production workloads in hours. If documents are already labeled using other tools, users can simply import labels with JSON in the Document format. Users can initiate training with a click of a button. Once the user has trained a model, they can auto-label documents to build a more robust training dataset to improve model performance.
Evaluate a model and iterate
Once a model is trained, it’s time to evaluate it by looking at the performance metrics–F1 score, precision, recall, etc. Users can dive into specific instances where the model predicted an error, then provide additional training data to improve future performance.
Going into production
Once a model meets accuracy targets, it’s time to deploy into production, after which the model endpoint can be called to classify document types.
Pub/Sub schemas are designed to allow safe, structured communication between publishers and subscribers. In particular, the use of schemas provides that guarantee that any message published adheres to a schema and encoding, which the subscriber can rely on when reading the data.
Schemas tend to evolve over time. For example, a retailer is capturing web events and sending them to Pub/Sub for downstream analytics with BigQuery. The schema now includes additional fields that need to be propagated through Pub/Sub. Up until now Pub/Sub has not allowed the schema associated with a topic to be altered. Instead, customers had to create new topics. That limitation changes today as the Pub/Sub team is excited to introduce schema evolution, designed to allow the safe and convenient update of schemas with zero downtime for publishers or subscribers.
Schema revisions
A new revision of schema can now be created by updating an existing schema. Most often, schema updates only include adding or removing optional fields, which is considered a compatible change.
All the versions of the schema will be available on the schema details page. You are able to delete one or multiple schema revisions from a schema, however you cannot delete the revision if the schema has only one revision. You can also quickly compare two revisions by using the view diff functionality.
Topic changes
Currently you can attach an existing schema or create a new schema to be associated with a topic so that all the published messages to the topic will be validated against the schema by Pub/Sub. With schema evolution capability, you can now update a topic to specify a range of schema revisions against which Pub/Sub will try to validate messages, starting with the last version and working towards the first version. If first-revision is not specified, any revision <= last revision is allowed, and if last revision is not specified, then any revision >= first revision is allowed.
Schema evolution example
Let’s take a look at a typical way schema evolution may be used. You have a topic T that has a schema S associated with it. Publishers publish to the topic and subscribers subscribe to a subscription on the topic:
Now you wish to add a new field to the schema and you want publishers to start including that field in messages. As the topic and schema owner, you may not necessarily have control over updates to all of the subscribers nor the schedule on which they get updated. You may also not be able to update all of your publishers simultaneously to publish messages with the new schema. You want to update the schema and allow publishers and subscribers to be updated at their own pace to take advantage of the new field. With schema evolution, you can perform the following steps to ensure a zero-downtime update to add the new field:
1. Create a new schema revision that adds the field.
2. Ensure the new revision is included in the range of revisions accepted by the topic.
3. Update publishers to publish with the new schema revision.
4. Update subscribers to accept messages with the new schema revision.
Steps 3 and 4 can be interchanged since all schema updates ensure backwards and forwards compatibility. Once your migration to the new schema revision is complete, you may choose to update the topic to exclude the original revision, ensuring that publishers only use the new schema.
These steps work for both protocol buffer and Avro schemas. However, some extra care needs to be taken when using Avro schemas. Your subscriber likely has a version of the schema compiled into it (the “reader” schema), but messages must be parsed with the schema that was used to encode them (the “writer” schema). Avro defines the rules for translating from the writer schema to the reader schema. Pub/Sub only allows schema revisions where both the new schema and the old schema could be used as the reader or writer schema. However, you may still need to fetch the writer schema from Pub/Sub using the attributes passed in to identify the schema and then parse using both the reader and writer schema. Our documentation provides examples on the best way to do this.
BigQuery subscriptions
Pub/Sub schema evolution is also powerful when combined with BigQuery subscriptions, which allow you to write messages published to Pub/Sub directly to BigQuery. When using the topic schema to write data, Pub/Sub ensures that at least one of the revisions associated with the topic is compatible with the BigQuery table. If you want to update your messages to add a new field that should be written to BigQuery, you should do the following:
1. Add the OPTIONAL field to the BigQuery table schema.
2. Add the field to your Pub/Sub schema.
3. Ensure the new revision is included in the range of revisions accepted by the topic.
4. Start publishing messages with the new schema revision.
With these simple steps, you can evolve the data written to BigQuery as your needs change.
Quotas and limits
Schema evolution feature comes with following limits:
20 revisions per schema name at any time are allowed.
Each individual schema revision does not count against the maximum 10,000 schemas per project.
Additional resources
Please check out the additional resources available at to explore this feature further:
The largest gathering of data and analytics leaders in North America is happening March 20 – 22nd in Orlando, Florida. Over 4,000 attendees will join in person to learn and network with peers at the 2023 Gartner® Data & Analytics Summit. This year’s conference is expected to be bigger than ever, as is Google Cloud’s presence!
We simply can’t wait to share the lessons we’ve learned from customers, partners and analysts! We expect that many of you will want to talk about data governance, analytics, AI, BI, data management, data products, data fabrics and everything in between!
We’re going big!
That’s why we’ve prepared a program that is bound to create opportunities for you to learn and network with the industry’s best data innovators. Our presence at this event is focused on creating meaningful connections for you with the many customers and partners who make the Google Cloud Data community so great.
We’ll kick off with a session featuring Equifax’s Chief Product & Data Analytics Officer, Bryson Koehler and Google Cloud’s Ritika Gunnar. Bryson will share how Equifax drove data transformation with the Equifax Cloud™. That session is on Monday, 3/20 at 4pm. After you attend it, you will realize why Bryson’s team earned the Google Cloud Customer of the Year award twice!
On Tuesday, you’ll have at least 4 opportunities to catch me and the rest of the team:
At 10:35am, Starburst’s Head of Product, Vishal Singh & I will cover how companies can turn Data Into Value with Data Products. We’ll discuss the maturity phases organizations graduate through and will even give you a demo live!
At 12:25pm, our panel of experts, LiveRamp’s Kannan D.R & Quantum Metric’s Russell Efird will join Google Cloud’s Stuart Moncada to discuss how companies can build intelligent data applications and how our “Built with BigQuery” program can help your team do the same.
And if all of this is not enough you will find some of our partners present inside the Google Cloud booth (#434). LiveRamp, Neo4j, Nexla, Quantum Metric, and Striim have all prepared innovative lighting talks that are bound to make you want to ask questions.
There are over 900 software companies who have built data products on our platform and while you don’t have 900 sessions at the event (we tried!), you can stop by our booth to inquire about the recent integrations we announced with Collibra, Elastic, MongoDB, Palantir, ServiceNow, Sisu, Reltio and more!
Top 5 Gartner sessions
I can’t wait to see all of you in person and our team looks forward to hearing how we can help you and your company succeed with data.
Beyond the above, there are of course many Gartner sessions that you should put on your schedule. In my opinion, there are at least 5 you can’t afford to miss:
Financial Governance and Recession Proofing Your Data Strategy with Adam Ronthal
What You Can’t Ignore in Machine Learning with Svetlana Sicular. I still remember attending her first session on this topic years ago — it’s always full of great stats and customer stories.
Ten Great Examples of Analytics in Action with Gareth Herschel. If you’re looking for case studies in success, sign up for this one!
Ask the Experts series, particularly the one on Cost Optimization with Allison Adams
Data Team Organizations and Efficiencies with Jorgen Heizenberg, Jim Hare and Debra Logan,
I hope you’ve found this post useful. If there is anything we can do to help, stop by the Google Data Cloud booth (#434).
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
You’re in the process of hiring another copywriter for your business, and you had a colleague tell you to try out AI-generated content. AI-generated content has been all the buzz lately as technology gets increasingly smarter, so it’s hard to tell when it’s appropriate and effective to use it. As of last month, over 200 ebooks on Amazon listed ChatGPT as an author or co-author.
AI content has helped a number of businesses. While major technology companies like Amazon have obviously used AI for years, smaller companies use it as well. Jeremiah Campbell, an owner of Brickworks Property Restoration, has reported using AI as well.
AI technology has helped us push our digital marketing efforts into overdrive and automate many tasks that we used to outsource. This is important for small businesses trying to manage costs during a recession.
There are plenty of positives when it comes to using AI-generated content for your business. After all, it’s a helpful tool that’s designed to perform exactly what you tell it to. There’s a good reason that many businesses are starting to make use of AI tools to help them with their content.
The top 3 pros for using AI-generated content:
Increased efficiency
Cost-effective
Consistent in tone and style
Increased Efficiency
One of the major advantages of using AI-generated content for your business is that it increases efficiency. While it may take a team member an hour to complete a blog post, an AI can perform the task in minutes. In turn, this saves businesses time and resources that can be better spent on tasks and functions that require human intervention and focus.
Cost-Effective
If you’re looking to save on costs with your business, using AI-generated content can help save you quite a bit of money. Making use of AI-generated content means you don’t have to hire out for content – or at the very least, you don’t need to have as many employees. This is why one of the biggest benefits of using AI-generated content is that it’s cost-effective.
Consistent in Tone and Style
Another helpful benefit of AI-generated content is that it can be incredibly consistent with tone and style. Consistency is important when it comes to content, and sometimes it can be more challenging for a human to replicate the same style and tone throughout a piece.
Cons of AI-Generated Content
While there are plenty of positive benefits to using AI-generated content for your business, there are also some downsides to it. It’s important to keep in mind what content you’re wanting to generate as there are some pieces that don’t do well with an AI conjuring it up.
Top 3 cons for using AI-generated content:
Lacks originality and creativity
Usually requires human intervention
Impersonal in nature
Lacks Originality and Creativity
AI-generated content is great at putting together pieces of content, but don’t expect it to create something original or creative. Seeing as AI-generated content uses information already available to it, it’s impossible for it to be innovative in its own right. This could shine through in the content it creates for you – and not in a good way.
Usually Requires Human Intervention
If you decide to use AI-generated content for your business, you’ll most likely need at least one person who edits and revises the content. You don’t want your content to look as though it’s written by a machine, and this can only be corrected if a human has hands on it.
Impersonal in Nature
AI-generated content isn’t one-size-fits-all. There are plenty of pieces of content that it just cannot perform effectively or accurately, particularly due to its impersonal nature. It’s important to keep in mind that a machine can never entirely replicate a human voice.
We have seen some massive changes with AI technology in the past year, especially in the field of marketing. While we have talked about advances in AI for years on this blog, the rest of the world is finally coming to grips with the extent of the AI revolution.
This past week, a growing number of people have talked about the changes that ChatGPT 4 is brining to our lives. There are even concerns that ChatGPT can encourage cheating in college, because it is capable of passing the SAT and bar exams.
While AI is changing many industries, digital marketing is among those most affected. A number of AI tools have made it easier for digital marketers to do their jobs effectively.
What Are the Most Prominent AI Tools Affecting Digital Marketing in 2023?
In 2023, the digital marketing landscape is evolving rapidly. Businesses must use AI to stay ahead of the curve to capitalize on their full potential. Providing abundant, exciting opportunities for creativity and growth is a digital marketer’s most rewarding experience and new advances in AI technology make it easier than ever.
And behind every great campaign lies plenty of administrative work, which, luckily enough, has been made easier with numerous tools that offer exceptional support in streamlining your entire lifecycle. Here are the best marketing tools in 2023 that will help level up your game.
ChatGPT
ChatGPT is an AI content writing tool that can help digital marketers create new content more quickly than ever. Companies can use ChatGPT to create blog posts, email series, social media posts and much more.
Search engine optimization: Semrush
Search engine optimization (SEO) is still one of the top marketing tools in any successful digital marketing campaign. SEO ensures that your website appears near the top of search engine results when potential customers type in relevant keywords or phrases. Semrush is one of the best SEO tools available today. It provides users with powerful analytics that allow them to track their progress and make data-driven decisions on improving their rankings. In addition, Semrush provides users valuable insights into their competitors’ strategies, allowing them to stay one step ahead of their competition.
Earlier this year, SEMrush published a blog post talking about the best AI tools of 2023. They mentioned their own AI tool, which is titled Semrush SEO Writing Assistant. This tool allows users to enter their chosen keywords directly in the SEMrush interface and it helps users create their own content that is geared towards SEO.
Social media: Hootsuite
Hootsuite is widely considered one of the best marketing tools for digital marketers in 2023 and beyond. This platform has a number of AI features that help digital marketers. They developed a tool called Hootbot, which is a fully automated social AI tool. Hootsuite can also be integrated with Lately, which is an AI content creation tool.
Using Hootsuite’s AI features allows you to easily manage multiple social media accounts via one central dashboard, letting you seamlessly schedule posts, share content, and track analytics on all platforms. The platform’s integrated analytics provide rich data and insights into your performance on each account, so you can fine-tune your strategies to get the most out of your social campaigns. Using Hootsuite makes staying productive across multiple channels easier than ever, enabling you to maximize the returns of your efforts with minimal effort.
Analytics: Google Analytics
Having accurate data is essential to understanding how your digital marketing efforts are performing as time goes on. Google Analytics provides an incredibly powerful analytics platform, giving you the information needed to assess the performance of each page on your website across multiple areas. For example, analytics can tell you where people come from, which pages have lower conversion rates, or any other trends that need attention. This means you don’t have to rely on guesswork or assumptions about your website traffic; instead, you can make informed decisions based on real insights.
Lead generation and capture: Salesforce Essentials
Lead generation and capture are important components of successful digital marketing efforts as they help you build relationships with potential customers who may eventually become paying clients or loyal followers. Salesforce Essentials offers users powerful lead capture capabilities that allow them to collect information on potential leads such as name, e-mail address, phone number, and many more, while also giving them access to detailed analytics regarding those leads so they can better understand who they should reach out too first or focus more resources on certain types of leads over others.
Use AI Tools Can Help Drastically Improve Your Digital Marketing Efforts
AI technology has significantly changed the state of digital marketing. Finding the right marketing tool for your business doesn’t have to be challenging. Start by asking yourself what you wish to achieve and pinpointing essential features when achieving those goals. No matter what form of current promotion is needed most, rest assured there are plenty of high-quality AI solutions at hand.
Many different industries are growing due to the proliferation of big data. The dropshipping industry is among them.
Paul Glen of IBM’s Business Analytics wrote an article titled “The Role of Predictive Analytics in the Dropshipping Industry.” Glen shares some very important insights on the benefits of utilizing predictive analytics to optimize a dropshipping commpany.
Glen states that a data review can help you understand if your dropshipping company is profitable. Data reviews can also give you insights into what products customers prefer, aiding product making and curation decisions. A data analysis can also answer important questions about your company’s financial health like: how much income was earned this quarter and where does most of your traffic come from? Understanding the data is key, but also knowing what to do with the data gleaned from it.
Glen points out that dropshipping companies account for 33% of all ecommerce sales. Therefore, entrepreneurs that use predictive analytics strategically can earn a large share of this growing market.
New Dropshipping Entrepreneurs Should Utilize Predictive Analytics to Develop a Competitive Edge
Surprisingly, dropshipping is a unique type of fulfillment business model in which you can start with the minimum and even no money at all. You can start dropshipping as a part-time business at any age and organize a successful business that can include other business models. The main reason why dropshipping is a perfect start for entrepreneurs is that the dropshipper does not have to produce or store any goods. The main dropshipper’s task is to organize an online attraction platform where customers order goods. These orders are sent to the actual Manufacturer by dropshipper. The Manufacturer can execute the packing and delivery and minimize the list of dropshipper’s tasks.
Unfortunately, creating a successful dropshipping business is easier said than done. The good news is that new advances in predictive analytics can help companies develop an edge.
So, let’s dig into details if dropshipping is a reality with null investments and how to start without money and use predictive analytics to have an edge over your competition.
Use predictive analytics to organize your dropshipping business
The success of dropshipping is in three aspects which depend on a dropshipper:. Predictive analytics can help in a number of ways, including the following
Predictive analytics can help you choose the right product. New AI algorithms can evaluate a number of different economic and market variables to gauge the future demand for various products, which makes it easier for dropshippers to succeed.
TechTarget’s Mary Pratt points out that predictive analytics can help you promote your online store and your products (the price of a product consists of 25% actual product price and 75% of the advertising campaign). Predictive analytics employs multiple methods to uncover patterns in existing and past data. This allows companies to comprehend facts from the past, establish why things occurred, anticipate what could occur, and finally decide on the best way to optimize outcomes. This allows them to better position their online ads and organic marketing efforts.
Predictive analytics can help you choose the right manufacturer, wholesaler and sales partners. Many predictive analytics models can evaluate likely outcomes of different partners based on variables showing which types of companies have been successful in the past.
You can successfully structure dropshipping businesses can be performed on different platforms and predictive analytics tools can help you succeed. But since we are talking about free methods, the most popular commerce platform Shopify will not be discussed here, as it has a monthly fee. Some time-consuming steps can be improved through numerous paid services that you may use later after you receive profit from your first orders.
The list below is a simple guide to becoming a dropshipper without investments.
1. Find your niche and winning product.
The first stage of the process is to find the best products to promote. Predictive analytics technology makes this process much easier.
For this task, you need only a computer and an Internet connection. You can research social networks, AliExpress, the most popular requests in the search engines, Amazon, news trends, and all other available resources to discover the winning dropshipping product. In addition, you can try a free trial version of any dropshipping product research tool that uses predictive analytics technology to minimize the most time-consuming and challenging preparation step. Typically, the product has a price of less than $200. The top niche categories for recent years were: beauty, pets, technology, and kids.
2. Find a Manufacturer/Wholesaler/ Seller (Supplier).
For this task, you need the same resources as for task one. You need to find a Company that is willing to work with dropshipper. It can be a Chinese manufacturer or a local company that can’t perform sales alone. It also can be wholesalers with excess goods they want to sell with your help. You need to conclude with your Supplier a contract/ agreement with all terms of production, delivery, returns, and refunds. You should agree to pay for the goods you perform after the Customer pays you, as you do not have initial money for advance payment.
You can use predictive analytics tools that determine whether a company will do a good job or not based on reviews and other key factors. However, it is important to do your research instead of relying entirely on your predictive analytics model. You need to always use common sense when choosing a business partner, but the predictive analytics model can at least help you create a good short list of partners.
3. Organize the online store.
If you already have popular social networks and websites with many subscribers, you can start selling products there. For other dropshippers, you need to find a free platform (like Bigcartel). If you want to sell many products, you need to find a tool that will reprice the products from the Suppliers price to yours, including a dropshipping fee. A dropshipping fee is your profit, which you set individually. Usually, free platforms allow you to have a limited number of products, from five to ten, but you can organize several stores.
You need to create a colorful presentation of products to attract Customer’s attention. And for sure, you need to create your logo and find an excellent unique brand name; there are a lot of free tools (business name generators and logo creators).
You can use data analytics to improve the success of your store down the road. Many platforms allow you to integrate Google Analytics or other analytics solutions.
4. Make the business legal.
You must open a bank account and determine the customer payment method (Visa, MasterCard, Stripe, or PayPal). Also, remember to pay all taxes and become the official entrepreneur in your country based on local codes.
5. Start promotion company
After you have made all the preparation steps and are ready to open your online store, you can start an advertising campaign to attract customers. Again, there are a lot of free methods:
Actively use your social networks.
Make videos.
Communicate with bloggers and organize barter.
Do guest blogging.
Comment on popular forums.
If you have a website, you can use numerous free SEO tools to improve your position in search engines. It would be best to focus on proper keywords and backlinks, make proper metadata, take care of the time required to download a webpage, etc. You can find the complete list of advice for sure you can receive only with paid tools, but by using several free versions, you still can achieve a high SERP.
Order fulfillment
If you perform all steps correctly, after step 5, you will receive the first orders. You will not face any expenses for order fulfillment, again only PC or even smartphone.
Typically, order fulfillment goes the following way:
You receive the order from the Customer.
You send the other to the Supplier.
You inform the Customer about the delivery date.
The supplier produces the goods and delivers them to the Customer.
You receive feedback from the Customer about successful/unsuccessful experiences.
As the Customer does not have direct contact with the Supplier, all questions regarding production and delivery should be on dropshipper (customer service). Moreover, all complaints and refunds are the tasks of the dropshipper. So, it is essential to have constant communication with your Customers to receive positive reviews and promote your website.
As you can see, predictive analytics technology plays a crucial role in helping dropshipping companies succeed. Dropshipping is a perfect opportunity to work anywhere and not deal with inventory and shipping products. Why not take advantage of the right analytics tools to get off on the right foot?
You may start a business without investments and can try different products before you find the best one. There are no storage, production, or staff expenses; you can perform all tasks yourself unless you expand your business to a large market.
As a middleman, you have fewer risks than a Manufacturer, but you receive a lot of knowledge on how to deal with Customers, Delivery Companies, and Suppliers. With all this experience, you can invest in addition to your profit in another type of business or expand your dropshipping website to a global market. So remember to invest your profit into marketing.
As you see, you can take a few simple steps of dropshipping business, creating several online shops simultaneously with the use of predictive analytics technology. There are no limits in dropshipping business!
