Thursday, January 2, 2014

Optimizing Star Queries

You should consider the following when using star queries:

Tuning Star Queries

To get the best possible performance for star queries, it is important to follow some basic guidelines:
  • A bitmap index should be built on each of the foreign key columns of the fact table or tables.
  • The initialization parameter STAR_TRANSFORMATION_ENABLED should be set to true. This enables an important optimizer feature for star-queries. It is set to false by default for backward-compatibility.
  • The cost-based optimizer should be used. This does not apply solely to star schemas: all data warehouses should always use the cost-based optimizer.
When a data warehouse satisfies these conditions, the majority of the star queries running in the data warehouse will use a query execution strategy known as the star transformation. The star transformation provides very efficient query performance for star queries.

Using Star Transformation

The star transformation is a powerful optimization technique that relies upon implicitly rewriting (or transforming) the SQL of the original star query. The end user never needs to know any of the details about the star transformation. Oracle's cost-based optimizer automatically chooses the star transformation where appropriate.
The star transformation is a cost-based query transformation aimed at executing star queries efficiently. Oracle processes a star query using two basic phases. The first phase retrieves exactly the necessary rows from the fact table (the result set). Because this retrieval utilizes bitmap indexes, it is very efficient. The second phase joins this result set to the dimension tables. An example of an end user query is: "What were the sales and profits for the grocery department of stores in the west and southwest sales districts over the last three quarters?" This is a simple star query.

Star Transformation With a Bitmap Index

A prerequisite of the star transformation is that there be a single-column bitmap index on every join column of the fact table. These join columns include all foreign key columns.
For example, the sales table of the sh sample schema has bitmap indexes on the time_idchannel_idcust_idprod_id, and promo_id columns.
Consider the following star query:
SELECT ch.channel_class, c.cust_city, t.calendar_quarter_desc,
   SUM(s.amount_sold) sales_amount
FROM sales s, times t, customers c, channels ch
WHERE s.time_id = t.time_id
AND   s.cust_id = c.cust_id
AND   s.channel_id = ch.channel_id
AND   c.cust_state_province = 'CA'
AND   ch.channel_desc in ('Internet','Catalog')
AND   t.calendar_quarter_desc IN ('1999-Q1','1999-Q2')
GROUP BY ch.channel_class, c.cust_city, t.calendar_quarter_desc;

Oracle processes this query in two phases. In the first phase, Oracle uses the bitmap indexes on the foreign key columns of the fact table to identify and retrieve only the necessary rows from the fact table. That is, Oracle will retrieve the result set from the fact table using essentially the following query:
SELECT ... FROM sales
WHERE time_id IN
  (SELECT time_id FROM times 
   WHERE calendar_quarter_desc IN('1999-Q1','1999-Q2'))
   AND cust_id IN
  (SELECT cust_id FROM customers WHERE cust_state_province='CA')
   AND channel_id IN
  (SELECT channel_id FROM channels WHERE channel_desc IN('Internet','Catalog'));

This is the transformation step of the algorithm, because the original star query has been transformed into this subquery representation. This method of accessing the fact table leverages the strengths of Oracle's bitmap indexes. Intuitively, bitmap indexes provide a set-based processing scheme within a relational database. Oracle has implemented very fast methods for doing set operations such as AND (an intersection in standard set-based terminology), OR (a set-based union), MINUS, and COUNT.
In this star query, a bitmap index on time_id is used to identify the set of all rows in the fact table corresponding to sales in 1999-Q1. This set is represented as a bitmap (a string of 1's and 0's that indicates which rows of the fact table are members of the set).
A similar bitmap is retrieved for the fact table rows corresponding to the sale from 1999-Q2. The bitmap OR operation is used to combine this set of Q1 sales with the set of Q2 sales.
Additional set operations will be done for the customer dimension and the product dimension. At this point in the star query processing, there are three bitmaps. Each bitmap corresponds to a separate dimension table, and each bitmap represents the set of rows of the fact table that satisfy that individual dimension's constraints.
These three bitmaps are combined into a single bitmap using the bitmap AND operation. This final bitmap represents the set of rows in the fact table that satisfy all of the constraints on the dimension table. This is the result set, the exact set of rows from the fact table needed to evaluate the query. Note that none of the actual data in the fact table has been accessed. All of these operations rely solely on the bitmap indexes and the dimension tables. Because of the bitmap indexes' compressed data representations, the bitmap set-based operations are extremely efficient.
Once the result set is identified, the bitmap is used to access the actual data from the sales table. Only those rows that are required for the end user's query are retrieved from the fact table. At this point, Oracle has effectively joined all of the dimension tables to the fact table using bitmap indexes. This technique provides excellent performance because Oracle is joining all of the dimension tables to the fact table with one logical join operation, rather than joining each dimension table to the fact table independently.
The second phase of this query is to join these rows from the fact table (the result set) to the dimension tables. Oracle will use the most efficient method for accessing and joining the dimension tables. Many dimension are very small, and table scans are typically the most efficient access method for these dimension tables. For large dimension tables, table scans may not be the most efficient access method. In the previous example, a bitmap index on product.department can be used to quickly identify all of those products in the grocery department. Oracle's cost-based optimizer automatically determines which access method is most appropriate for a given dimension table, based upon the cost-based optimizer's knowledge about the sizes and data distributions of each dimension table.

The specific join method (as well as indexing method) for each dimension table will likewise be intelligently determined by the cost-based optimizer. A hash join is often the most efficient algorithm for joining the dimension tables. The final answer is returned to the user once all of the dimension tables have been joined. The query technique of retrieving only the matching rows from one table and then joining to another table is commonly known as a semi-join.

Wednesday, October 30, 2013

missing expression error while running ODI Interface.


Scenario:
Sometimes when ODI interface is run, we may encounter an error like below

Caused By: java.sql.SQLSyntaxErrorException: ORA-00936: missing expression

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:462)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:405)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:931)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:481)


If we check the code, it generates a code like below

 select 1 from APPS.MTH_ORGANIZATIONS_L T
where  
and ((T.ORGANIZATION_NAME = S.ORGANIZATION_NAME) )

here after WHERE there should have been (1=1) followed by AND condition.

Reason:
This occurs if none of the target columns have been defined as a key.

Solution:
Make one of the columns in the target table as Key by check the box. Now run the interface again. It should solve the issue.



Tuesday, October 29, 2013

Change Password of BISystemUser

Changing a password of BISystem user involves two steps.

1. Change password in /console
2. Change password in /em.

Change password in /console

1. Stop the system components in Enterprise Manager.
Click on Business Intelligence >Core application> Availability
 
 
2. Log into Weblogic Console and change the BISystemUser password.
Click on security realms > myreams > user and group
 
BISystemUser > Passwords
 
 
 
Change password in /em
3. Change password in EM:
Weblogic Domain > right click on bifoundation_domain > Security > Credentials > oracle.bi.system > system.user > Edit > change the password
 

 
4. Start BI System components from Enterprise Manager.
Click on Business Intelligence >Core application> Availability

5. Wait for 10 mins
6. Try the new password in the OBIEE URL.

Monday, October 28, 2013

Weblogic Domain Creation Fails due to PermGen Space Error OBIEE 11.1.1.6.0


During installation of OBIEE 11.1.1.6.0 on Windows Server 2008 - 64bit, it failed at the Domain Creation Step(Step 13 of 14) with the below error code

Creating a new AdminServer Object ...

 AdminServer port is 7001 Starting the domain ...

 java.lang.OutOfMemoryError: PermGen space
 java.lang.OutOfMemoryError: PermGen space
 java.lang.OutOfMemoryError: PermGen space

Reason:
This occurs due to insufficient runtime memory to JAVA during installation.

Solution:
This can be avoided by modifying the ORAPARAM.ini file in installer location like below.
D:\bishiphome\Disk1\install\win64\ORAPARAM.ini

Add the below line
JRE_MEMORY_OPTIONS=" -mx1024m -XX:MaxPermSize=1024m"

First make a clean uninstall of the entire OBIEE including the files from the directories.
Then restart the installation

Now during the installtion, at the stage when OneOff Patches are being installed(Step 12 of 14), make a modification to the ORAPARAM.ini parameter in the installed location like below.
D:\OBIEEMIDDLEWARE\Oracle_BI1\oui\ORAPARAM.ini

Add the same line specified above.

This would be helping in allocating sufficient memory to JAVA when
D:\OBIEEMIDDLEWARE\Oracle_BI1\bin\config.bat file is run during next step(Step 13 of 14)




Default Users Setup in OBIEE 11.1.1.6.0



Default Users Setup that comes with OBIEE 11.1.1.6.0

User
Groups
BISystemUser
Administrators
Weblogic
Administrators

BIAdministrators
OracleSystemUser
OracleSystemGroup

Default Application Role Setup OBIEE 11.1.1.6.0


The below table represents the default setup of Application Roles. This is just to revert your settings just in case we end up corrupting the Setup.

Default Application Role Setup.

Role Name
Display Name
Principal
Type
BISystem
BI System Role
BISystemUser
User
BIAdministrator
BI Administrator Role
BIAdministrators
Group
BIAuthor
BI Author Role
BIAuthors
Group


BIAdministrator
Application Role
BIConsumer
BI Consumer Role
BIConsumers
Group


BIAuthor
Application Role


Authenticated-user
Authenticated Role