Good news, Business Architects! AI is not going to take over your jobs anytime soon. In this session, I aimed to create a business architecture information concepts table, which, while doable, has several challenges. While AI, at this stage, cannot replace human intelligence and expertise, it certainly does provide a helping hand, albeit one that needs close supervision. So let's get started...
Business Architecture Information Concepts: A Closer Look at ChatGPT's Abilities and Limitations
To help with the research and building the business architecture, I will again use the fictional Supreme Coffee case study.
What is a business architecture information concept?
BIZBOK's Information Concepts are important components of the Business Architecture knowledgebase because they provide a structured way to connect business concepts with data. These ideas serve as building blocks, transforming the organization's vision and mission into distinct elements. They are critical for identifying, utilising, and retaining the appropriate data throughout the organisation.
These concepts provide a solid foundation for Supreme Coffee, a local shop planning to franchise globally. The value of Information Concepts is derived from their interconnection with the capabilities, stakeholders, and business objects of the business.
Data collection was inefficient in the past due to its singular, point-by-point approach. BIZBOK improves on this by making Information Concepts tangible and facilitating discussion about them, allowing for a mutual understanding of business objects and relationships.
In essence, Information Concepts provide a formal framework for representing critical business information, thereby supporting activities such as data requirements alignment and service design. Their successful implementation promotes the use of accurate, efficient, and cohesive information, which is essential for a company's growth and competitiveness.
When trying to get the most out of information concepts, I decided to add some more concepts into the table I was building. This level of complexity was where the rubber hit the road with ChatGPT. It really struggled to bring these concepts together. So, as mentioned, this is where to need a Business Architect. Here's how I changed and worked these fields into the table for Supreme Coffee. (Note I have transposed the fields for easier viewing.)
| Column Heading | Description / Example |
|---|---|
| Unique ID | Serves as a unique identifier for each row to help with traceability |
| Capabilities | Levels 1, 2, and 3 represent a hierarchy of business capabilities |
| Business Objects | Representation of a "thing" in the business domain, like 'Coffee', 'Customer', 'Barista', 'Franchisee', 'Supply Chain', 'Sales' |
| Information Concept | Refers to a particular business concept, e.g., 'Franchisee Information' groups together related business objects like 'Franchisee Location', 'Franchisee Sales' etc. |
| Information Concept Category | Can be classified as either Primary or Secondary, like 'Franchisee Information' can be placed under a broader 'Franchise Operations' category |
| Information Concept Definition | Provides a deeper understanding of the concept |
| Information Concept Type | Used to specify different categories and hierarchies, e.g., Market Research, Brand Identity related to Brand Strategy Management |
| Related Information Concepts | Shows the relation of the main concept to other information concepts |
| Information Concept States | The concept can exist in different states such as 'Prospective', 'Active', 'Inactive', or 'Terminated', depending on the franchisee's current contractual status |
ChatGPT and Business Architecture: Findings and Observations
ChatGPT, like other AI products, has both advantages and disadvantages. I ran an experiment to see how it performed when creating an information concepts table, and here's what I discovered:
Lack of Deep Understanding:
The AI struggled with understanding the complexities of various data types, as well as the hierarchical levels of business capabilities. This led to some mistakes and inconsistencies within the table. You have to watch it like a hawk.
Difficulty with Context and Integration:
To manage the information volume, the process of building the table chat by chat resulted in a loss of context. It was also clear that the AI struggled to integrate information across different levels of the table, necessitating manual intervention.
Replication of Errors:
The AI displayed a tendency to repeat certain mistakes even after multiple iterations, implying that it might not learn from previous errors within a single session. For example, I explained that it needs to understand that the information concept should have a primary and secondary option, corrected it the first time, and then changed it again later in the chat.
Challenges with Table Complexity:
As the complexity of the table increased, so did the AI's performance degrade. It struggled to handle a larger or more complex table, which resulted in a number of errors.
Difficulty Differentiating Information:
The AI also struggled to distinguish between primary and secondary information concepts, resulting in incorrect classifications.
Implications and Recommendations for Using ChatGPT in Business Architecture
Even with its limitations, AI, specifically ChatGPT, showcases its potential as a useful tool for business architects. It isn't ready to fully imitate a human business architect's refined understanding and decision-making capabilities but it can provide assistance in breaking down complex tasks and offer helpful insights.
Here are some ways to optimize AI usage in such complex tasks:
Structured Inputs and Clear Instructions:
AI performs best when given structured input and direct instructions. Overloading it with information can cause it to become confused. So one information concept at a time.
Continuous Monitoring and Intervention:
Despite AI's ability to automate some tasks, human intervention is required to correct errors and guide the process. I had to double-check the work and take appropriate notes all the time.
Iterative Refinement:
Expecting AI to be perfect on the first try is unrealistic. Gradually improving its output can lead to better results. This means you must look for ways to improve the outputs, such as switching from a table structure to a list view, reducing the data being analysed, and starting a new chat if you want additional information.
Trial and Error:
Different ChatGPT versions may produce different results. In terms of creating user stories, ChatGPT 3 outperformed ChatGPT 4, indicating that experimenting with different versions may be beneficial.
Cross-Verification:
To ensure relevance and accuracy, AI output must be aligned with business capability outcomes and user stories. Of course, you'll have to do this in real life as well.
Conclusion: The role of AI in Business Architecture
Hold your hat on this one. The essential purpose of information mapping is to simplify high-level abstract concepts into a more accessible, well defined form. Unfortunately, ChatGPT struggled to accomplish this task effectively. The key to success here lies in absolute accuracy, a criterion that wasn't consistently met. I found myself having to manually correct or input information due to recurring errors resulting in multiple chats.
Despite this, I can't deny the benefits of AI, as it saved me substantial research time. However, it's clear that a deep understanding of business architecture principles is necessary to effectively monitor and manage the process for optimal results. While AI, like ChatGPT, can serve as a valuable tool for Business Architects, it is far from being able to replace the role in the near future.
