Rules to Remember with XML

Rules to Remember

We have covered the basic knowledge of XML. Now there are some standard rules you need to follow when you work with XML files. Any validation of the rules will throw you an error by any standard XML processor including the one used by BI Publisher. Good news is, XML has only a few rules and all the rules actually make sense and most importantly they are easy enough to follow.

So here is a list of main rules to remember for you at least.

• All XML Elements must have a closing tag
• Element and Attribute names are case sensitive
• Element must be properly nested
• Documents must have a root element
• Attribute Values must be quoted
• White Space value is preserved
• Element/Attribute Values can contain letters, numbers, and other characters
• Element Name Must not start with a number or punctuation character
• Element Name Must not start with the letters ‘xml’ (or XML)
• Element Name cannot contain spaces

I’ve seen many users at our clients have encountered errors when they developed or run BI Publisher reports (RTF template or Data template) because of the non-compliance with the above rules. As you know currently we still need to develop the Data Template files manually and many of us uses text editors to edit the XML format files. And we make some typo, forgot single/double quotes, mistakenly put a space in the element name, or forgot to end the node with closing element. All of these cases will throw you out an error. Also there is another type of problem because of the strict rules listed above. For example when you are developing a RTF Template but couldn’t get any data in the output even though there should be a data and you have mapped the XML element data in the RTF Template. This can often be the case where you have typed the element name in lower case when the element name is presented in upper case in your XML data. Yes, XML Element name is case sensitive! So you need to make sure if the XML Element name is upper case or lower case (or combination) in the XML data file first then need to correctly type the name in your RTF Template.

Many cases when you start developing Data Template or RTF Template and get some strange error that you have no idea are caused by violations of the above rules. So let’s keep those rules in your mind when you work with Data Template and RTF Template.

XML Basics

Today, I’ll talk about the basic of XML as part of the ‘Before you start RTF Template’ series.

I’m sure you have already seen XML files and known what the files look like. Especially the BI Publisher technology is designed around XML so it’s almost impossible not to see XML files when you develop any report. However, some of you might not have spent a time to learn what really the XML is more than what it looks. So, let’s spend some time (not a lot, so don’t worry!) to visit the XML basic.

What is XML ?

XML stands for eXtensible Markup Language. It is a W3C standard, and it was designed to describe data and to focus on what data is while a similar language called HTML was designed to display data and to focus on how data looks. XML uses a Document Type Definition(DTD) or an XML Schema to describe the structure of XML documents. Now it’s more common to use XML schema than the DTD to describe the data. And BI Publisher also uses XML Schema when it’s needed.

What are they? (Terminology)

Instead of going through all the detail of XML, I’ll focus on what’s really important for BI Publisher report developers. First, what are those tags inside the XML file? When you open the XML files you can see a bunch of tags and all of them looks same. However, there is some difference among them and each of them has its own name. The names you want to remember are the followings.

- Element
- Attribute
- Value
- Namespace
- Root
- Parent
- Child
- Sibling
- CDATA

Element

Element name is actually the main part of the XML and it acts as something like metadata. For example with the above sample XML the is the element name. With BI Publisher you use this as a place folder name when you map into RTF Template.

Attribute

Attribute is an additional metadata that tag along with the Element and add more meaning to the Element. For example with the above sample XML, ‘age’ is the attribute, which describes age of the employee in the EMPLOYEE element. And you can have multiple attributes per Element.

And Element and Attribute are the main component of the XML and the backbone of the file.

Value

Value is literally the value. Both the Element and Attribute can have their own values. The Element value is surrounded by Element start tag and Element end tag. e.g. Peter. The attribute value must be presented with double or single quotes surrounded. e.g. Peter

Node

Node is similar to something like group, unit, or family. And node can contain another node or multiple nodes inside and each node can act as an unit inside the higher level of nodes. With above sample, DEPARMENT is a node and you can find EMPLOYEE node inside the DEPARTMENT node.

Namespace

Namespace is used to differentiate the elements that have same name but means different.

Root Element

Root is something like the ‘root’ that is used for Unix file system. It means a top level Element and there is only one in one XML file.

Parent/Child/Sibling

These are the terms to describe relationships of each element. And this acts as exactly same as your family hierarchy or file system. So the node that is at one level above from where you are is called Parent node while the ones that are at one level below are called Child node. With above sample ORACLE node is parent node for DEPARTMENT element and EMPLOYEE element is child node. And this is a relative concept, so while DEPARTMENT node is a child node for ORACLE node, DEPARTMENT node is at the same time a parent node for EMPLOYEE node.

CDATA

CDATA is used for a section for a comment out area, where you can type anything. Any XML standard processor will not process the content in this area. We’ll cover in the next section about XML standard rules but the content stored in this CDATA section doesn’t apply such rules so you can basically type anything here for your use. BI Publisher uses this area with Data Template to store SQL query.

Starting RTF Template Development

Anybody can start developing RTF Template with MS-Word right away. You download Template Builder, which is a MS-Word Add-in, and install it, then load a sample XML data file or connect to BI Publisher Enterprise Server, now you can use Template Builder’s UI wizard to construct tables, charts, or even cross tabs just a matter of 10 seconds or so. (I mean, just to start with!) From there you can start modifying the layout and formatting by just using MS-Word’s native functionalities that most of us are already familiar with. You can add page header/footer, change font size, color, or change table column size, etc. And by using Template Builder you can add even calculations pretty easily. So until this point everything looks just a piece of cake.

But the real challenge starts from here when you have some ‘complicated’ requirements that you didn’t think they would be that complicated. For example you might want to do a conditional formatting based on values inside the group that you’re working on or even outside the group. Or you might want to use some functions to process some data values in the way you need to use for another processing. Or you might need to customize the charts to meet your specific requirements. The Template Builder has been evolving for the last 2 years or so and adding many extra features that have reduced significant amount of manual customization work we needed before. However, we still need to perform some type of manual customization to meet real world requirements especially for pixel perfect type of reports and forms.

For such customization, you can double click a text form field, which opens a Template Builder dialog box, click on the ‘Advanced’ tab, type any code includes BI Publisher code (xdo) and XSL/XPATH/FO codes. Or for Chart customization, you can double click the chart image, which again opens MS-Word dialog box, and click on the ‘Web’ tab, now you can customize the chart definition XML with mix of XDO/XSL/FO codes.

This is where many developers stop and scratch their heads, and wonder what the hell is this ? And you take a look at BI Publisher’s User’s Guide, google some useful tips, and try & error, in order to achieve what it used to look simple to do once. And next time you realize it’s already a couple of weeks past and your project manager or customers keep asking you when they can be done or even if they can be really doable.

Welcome to the world of XML. All of the customization requires an understanding of XML technologies. Without a decent understanding of the technology you are going to just spend more time for nothing. However, with a correct and good amount of understanding of the technology the manual customization with XML codes doesn’t look that difficult anymore. In fact you can see the consistent logic behind it and can grasp easier. You can even find more functionalities with the XML technology, which enables you to enjoy much more flexible development of RTF Template.

Well, you don’t need to become a XML developer or anything. You just need to get a ‘decent’ amount of understanding. Or I would say just ‘Basic’ level of understanding first. And this basic understanding of XML technology makes a whole world of difference. I can tell you because that is what happened to me when I started working with this technology. This also happened to many of my colleagues from BI & Reporting background.

So from the next blog I’ll cover some of the basic XML technologies. If you can’t wait for the next blog, I’d recommend you to go ahead and check this W3School’s web site where you can find not only the overview of the technology also quick tutorials for you to learn.

The topics I’ll covering will be,
- XML
- XSL
- XPATH
- XSL-FO
-SQL
-DWH

Cheers!

Siebel BI Publisher

Business Process Management Life Cycle

Business Modeling Methods:

IDEF:
IDEF is an acronym for Integrated Definition.

IDEF0:
IDEF0 is a method used in Business Process Modeling, designed to model the decisions, actions and activities of an organization or system. Diagrams are based on simple box with arrow graphics and text labels.

IDEF3:
IDEF3 is a method used in Process Flow Modeling, designed to model processes and documenting the processes and it explains how a system, process or an organization works.

IDEF4:
IDEF4 is a method designed to model object oriented programming.

Data Modeling Methods:

IDEF1X(Integrated Definition for Information Modeling notation) and IE(Information Engineering notation)
IDEF1X and IE types are focused on the actual data elements present in a relational database. IDEF1X and IE use different symbols to represent entity and table relationships for designing relational databases.

Dimensional Modeling Notation:
This is a method used for designing data warehouses and data mart.

Business Process
Business processes are a set of activities involved within or outside an organization that work together to produce a business outcome for a customer or to an organization.

An organization may have specific goals, aims and targets to be achieved. For products to be manufactured or sold or serviced, there are several activities involved with this organization like employees recruitment, machineries installation, purchasing goods, manufacturing of products, selling and servicing the product to consumers/vendors, collecting the payments and implementing government’s rules etc., and these activities collectively are called as business processes.

Different kinds of Business Processes

Business Processes that are within an organization. (e.g. product manufacturing)

Business Processes that are between organizations e.g Mortgage sales by a mortgage company and escrow(payment of insurance and taxes) servicing for that mortgage done by some other company. These are called business to business transactions and are done through application interfaces.

Business Processes that are managerial.(HR and Recruitment process)

Business Processes that are operational.(service calls attended in call centers, information stored by bank tellers regarding opening an account etc.)

Business Processes that are activity oriented: The transformations that takes place in a data warehouse.

BPM Tools
Business Process Management(BPM) Tools are used to create an application that is helpful in designing business process models, process flow models, data flow models, rules and also helpful in simulating, optimizing, monitoring and maintaining various processes that occur within an organization.

Many tasks are involved in achieving a goal and these tasks are done either manually or with the help of software systems. For example, if an organization needs to buy a software application that costs 6 million dollars, then a request has to be approved by several authorities and managers. Here the request approval may be done manually or with the help of change management softwares(e.g. – Serena, Test Director); whereas when a person applies for a loan of three hundred thousand dollars, several internal and external human-centric or enterprise application-centric business processes are to be triggered in order to find out the details of that person before approving the loan. For these activities, BPM tool creates an application that co-ordinates the manual and automated tasks.

Apart from co-ordinating the processes, BPM tools also facilitate the capturing of the rules defined for processes, simulation of business processes, executing the processes in a sequential manner by workflow component, monitoring the business processes by Business Activity Monitoring component, creating score cards for key performance indicators and balancing score cards to help the executives in taking better decisions.

BI Reporting

Business Intelligence Tools Directory:

• Actuate
• Business Objects
• Cognos
• HummingBird
• Hyperion
• Information Builders
• MicroStrategy
• ProClarity
• Siebel

Business Intelligence

What is Business Intelligence?

Business Intelligence (BI) is a terminology representing a collection of processes, tools and technologies helpful in achieving more profit by considerably improving the productivity, sales and service of an enterprise. With the help of BI methods, the corporate data can be organized, analyzed in a better way and then converted into an useful knowledge of information needed to initiate a profitable business action. Thus its about turning a raw, collected data into an intelligent information by analyzing and re-arranging the data according to the relationships between the data items by knowing what data to collect and manage and in what context.

Importance of Business Intelligence:

A company’s collected raw data is an important asset where one can find solutions to many of an organisation’s critical questions like ‘what was the net profit for a particular product last year and what will be sales this year and what are the key factors to be focused this year in order to increase the sales?’. So there arises a necessity of a well planned BI system which can lead to a greater profitability by reducing the operating costs, increasing the sales and thereby improving the customer satisfaction for an enterprise.

With the help of a Business Intelligence System, a company may improve its business or rule over its competitors by exploring and exploiting its data to know the customer preferences, nature of customers, supply chains, geographical influences, pricings and how to increase its overall business efficiency.

Business Intelligence & Return on Investment (ROI):

Business Intelligence enables us to take some action based on the intelligence aquired using BI strategy. If this knowledge or information is not utilized properly in the right direction, there is no point in analyzing and finding the intelligence. For example, lets assume a company has implemented a BI system to analyze the customer interests and requirements enabling them to promote a particular product in the near future. All the analysis and knowledge management will be pointless and a waste of investment if they dont have a proper plan to approach the right customer at the right time. So Business Intelligence is all about strategies in increasing business efficiency while vastly cutting down the operating costs.

Implementing a Business Intelligence system in an organization requires a significant amount of money to be invested in order to build and implement a BI system and its applications. It requires more skilled top level managers to build a ROI(Return on Investment) model to analyze the costs involved in implementing and maintaining these BI models and methods to get the return on investment sooner.

A proper business action should be taken based on the strategies derived with the help of these intelligence models. Often an erraneous model and wrong assumptions can bring a loss much greater than building the entire Business Intelligence system itself. Once everything is done more properly in a way an organization want them to be, then the benefit that comes out of it is priceless.

BI Environment & Business Flow:

Business Intelligence is all about converting large amounts of corporate data into useful information, thereby triggering some profitable business action with the help of knowledge acquired through BI analysis. Implementing BI is a long process and it requires a lot of analysis and investment. A typical BI environment involves business models, data models, data sources, ETL, tools needed to transform and organize the data into useful information, target data warehouse, data marts, OLAP analysis and reporting tools.

Setting up a Business Intelligence environment not only rely on tools, techniques and processes, it also requires skilled business people to carefully drive these in the right direction. Care should be taken in understanding the business requirements, setting up the targets, analyzing and defining the various processes associated with these, determining what kind of data needed to be analyzed, determining the source and target for that data, defining how to integrate that data for BI analysis and determining and gathering the tools and techniques to achieve this goal.

Following sections explain each of these areas in detail and the sample figure shows a BI environment in its simplest form.

Data Warehousing Concepts

Several concepts are of particular importance to data warehousing. They are discussed in detail in this section.

Dimensional Data Model: Dimensional data model is commonly used in data warehousing systems. This section describes this modeling technique, and the two common schema types, star schema and snowflake schema.

Slowly Changing Dimension: This is a common issue facing data warehousing practioners. This section explains the problem, and describes the three ways of handling this problem with examples.

Conceptual Data Model: What is a conceptual data model, its features, and an example of this type of data model.

Logical Data Model: What is a logical data model, its features, and an example of this type of data model.

Physical Data Model: What is a physical data model, its features, and an example of this type of data model.

Conceptual, Logical, and Physical Data Model: Different levels of abstraction for a data model. This section compares and contrasts the three different types of data models.

Data Integrity: What is data integrity and how it is enforced in data warehousing.

What is OLAP: Definition of OLAP.

MOLAP, ROLAP, and HOLAP: What are these different types of OLAP technology? This section discusses how they are different from the other, and the advantages and disadvantages of each.

Bill Inmon vs. Ralph Kimball: These two data warehousing heavyweights have a different view of the role between data warehouse and data mart.

Data Ware housing Steps

After the tools and team personnel selections are made, the data warehouse project can begin. The following are the typical processes involved in the data warehousing project cycle.

• Requirement Gathering
• Physical Environment Setup
• Data Modeling
• ETL
• OLAP Cube Design
• Front End Development
• Report Development
• Performance Tuning
• Quality Assurance
• Rolling out to Production
• Production Maintenance
• Incremental Enhancements

Each page listed below represents a typical data warehouse phase, and has several sections:

• Task Description: This section describes what typically needs to be accomplished during this particular data warehouse phase.
• Time Requirement: A rough estimate of the amount of time this particular data warehouse task takes.
• Deliverables: Typically at the end of each data warehouse task, one or more documents are produced that fully describe the steps and results of that particular task. This is especially important for consultants to communicate their results to the clients.
• Possible Pitfalls: Things to watch out for. Some of them obvious, some of them not so obvious. However, all of them are real.

The Additional Observations section contains my own observations on data warehouse processes not included in any of the step.

Feel free to contact mehta.deepanshu@gmail.com about additional information regarding data warehousing that I should include.

Data Ware housing (DWH)

This Data Warehousing site aims to help people get a good high-level understanding of what it takes to implement a successful data warehouse project. A lot of the information is from my personal experience as a business intelligence professional, both as a client and as a vendor.

This site is divided into five main areas.

- Tools: The selection of business intelligence tools and the selection of the data warehousing team. Tools covered are:

• Database, Hardware
• ETL (Extraction, Transformation, and Loading)
• OLAP
• Reporting
• Metadata

- Steps: This selection contains the typical milestones for a data warehousing project, from requirement gathering, query optimization, to production rollout and beyond. I also offer my observations on the data warehousing field.

- Business Intelligence: Business intelligence is closely related to data warehousing. This section discusses business intelligence, as wellas the relationship between business intelligence and data warehousing.

- Concepts: This section discusses several concepts particular to the data warehousing field. Topics include:

• Dimensional Data Model
• Star Schema
• Snowflake Schema
• Slowly Changing Dimension
• Conceptual Data Model
• Logical Data Model
• Physical Data Model
• Conceptual, Logical, and Physical Data Model
• Data Integrity
• What is OLAP
• MOLAP, ROLAP, and HOLAP
• Bill Inmon vs. Ralph Kimball

- Glossary: A glossary of common data warehousing terms.

This site is updated frequently to reflect the latest technology, information, and reader feedback. Please bookmark this site now.