Showing posts with label Cognos. Show all posts
Showing posts with label Cognos. Show all posts

Monday, July 6, 2015

Using ODBO as datasource in Cognos TM1 Turbo Integrator (TI) Process

Need to copy TM1 cube data from one server to another ... read below.
In IBM Cognos TM1 Turbo Integrator (TI) Process, we can use ODBO as data source type.  Using ODBO we can connect to other OLAP cubes (ex: MS SSAS, TM1 cubes in other environment). ODBO is an abbreviation of abbreviation (OLE DB for OLAP - Object Linking and Embedding DataBase for OnLine Analytical Processing)
In this post, we will see how to use this ODBO in a TI process and copy a cube content from one TM1 server to another.  To use the ODBO, we need to provide following information in the TI:
  • ODBO Provider Name - From the drop down choose IBM Cognos TM1 OLEDB MD Providee
  • ODBO Data Source - Enter your Admin Host (IP address or the DNS Alias).  If there are multiple NIC adapters present on your server, then enter the IP address on which the traffic of TM1 is routed
  • ODBO Catalog - This is the TM1 server name.  In below example we are connecting to Planning Sample instance
  • ODBO User ID, ODBO Password - Enter the credentials required to connect to the TM1 server
  • Connection Parameters - Leave it blank if you are working with native authentication (i.e. mode is 1), else you will need to enter the namespace name
Below is a screen grab of these parameters filled in.  Click on Connect.  If the parameters passed are correct, then the "Load ODBO Cube" tab (highlighted in yellow) is activated.


Note: If you have TM1 installed on AIX then please read this IBM KB article (Ref # 7041662) to work with ODBO

Before we jump ahead, let me show the cube list from the 2 environment.  In the top TM1 server there are no cubes and dimensions.  We will therefore be importing plan_Report cube from Planning Sample.  We will build the TI process to not only copy the cube data but also build the dimensions in the 1st server.

After you have successfully established the ODBO connection, click on the tab Load ODBO Cube.  Select the cube from where you want to pick the data, and enter the name of the target cube, as shown below.  Since we do NOT have the target cube, we will pick the choose option of Create cube.  Since we are copying data from one cube to another, there is possibility of copying a very large number of records.  In such scenarios, we wouldn't want to enable the logging (as this will create additional traffic, increase tm1s.log file size and slow down the process).  Therefore the option of Enable cube Logging is unchecked.

Now, click on the next tab Cube Dimensions and then on MDX Query tab, they should look like this below.  Since none of the dimensions exist in the target environment, I have chosen the option of Create and no filter is selected.  Observe the MDX of the highlighted dimension.

 


























Let's go back to the Cube Dimensions tab and alter the filter on Departments dimension.  We could either select individual elements as shown



















With level 2 selected the MDX will now appear as shown below.  Notice the change in MDX between this and the one previously shown.  In fact, you can compare the MDX for rest of the dimensions vs this one.  You will notice that, in case of Department MDX is explicitly selecting Level2 members, while in the others, it is using ISLeaf function to evaluate.














Now Save the process as LoadCube_ODBO and close the TI Editor. Let's take a look at the processes list in the target environment.











Not only has it created the main process, but it also has created child processes for each of the dimension involved in the cube build.  If you were to look in the prolog of the main process, in our case "LoadCube_ODBO", you will see that it executes the child processes first, before beginning the main process!

 
Well, I guess the engine is smart enough to create the child processes and in turn use them in the main process.  One limitation feature is that, if you were to open the main process, don't do anything else, just do a save as, in our case let's save it as "LoadCube_New_ODBO" and refresh the architect screen.  You will notice there is only one new process added to the existing list, as seen below:











The next logical question is, whether in the prolog tab of the new process
  • Is it still calling the old processes of dimension build? OR
  • Is it calling the processes with new name in dimension build?
If it is the latter, then surely there is a problem, isn't it?  The processes with new name for dimension build don't exist yet.  So let's just click on the prolog code (don't do anything else), notice the prolog code




It looks like we have a problem, don't we?  The processes are using new TI names for dimension build and yet in the architect they are not there!  We are still not done yet :) ... We were in the prolog screen looking at the code; now, click on save button and check out architect.  The processes have now been magically renamed to use the new names !!!  This is the feature you got to be aware of.

 









 Happy ODBOing!








Thursday, June 25, 2015

IBM Cognos TM1 Import source in SPSS reads incorrect data from TM1 Cube - How to fix it?

Last week, I published a post on reading IBM Cognos TM1 data into SPSS Modeler, using the newly available source palette within SPSS Modeler called "IBM Cognos TM1 Import".  One of my colleagues posed a question, where in she was getting incorrect results using Cognos TM1 cube as source in SPSS.  Last digit of the number gets dropped off in SPSS Modeler!




If you are on SPSS Modeler ver 17, there is no need to fret.  This is seen only in SPSS Modeler Ver 16.  There is a well documented KB article on IBM Site - "IBM Cognos TM1 Import source node is reading continuous data incorrectly" (Reference #:1683459).  The article tells us to modify the Process ExportToSPSS to fix the issue
Value = if(VALUE_IS_STRING = 1, SVALUE, NumberToStringEx(NVALUE,'#####0.0##', '.', ','));
This post describes how to make this change in the process.  There are 3 ways to implement this fix:
  1. Use IBM Cognos TM1 Architect
  2. Use IBM Cognos TM1 Performance Modeler
  3. Using Text Editor

Text Editor

You would really need to know what you are doing here.  There are numbers present in the beginning of every line, which TM1 system uses to parse out the code in a TI.  A wrong modification will result in unexpected behavior of the program, including errors.  I would not recommend this approach!

TM1 Performance Modeler

By far the easiest way to fix the code, especially in the current scenario.  I will explain more about using Architect and the problems faced, later.  If you open TM1 Performance Modeler, locate the TI process, double click, edit and save.  That's it.  You are ready to use the modified code in SPSS Modeler.
While the change is easy and swift to make, adding an extra line of code in TM1 Performance Modeler increased the file size from 55KB to 126KB !


TM1 Architect

If you open TM1 Architect, logon to the server, locate the TI and when you double click it, you get 2 error messages right away.  See the screens below:

The reason these messages pop up is because, this TI process uses IBM Cognos TM1 Cube View as data source and there is no cube view defined for it.:

If you click on Variables tab,you will notice there are 128 variables present.  So we will need to create a cube with 127 dimensions (yes that's right 127 dimensions, not 128).  128th variable will be the value (SValue or NValue) of the measure.


Using Architect to modify the process is not straightforward and will involve little bit of workaround.  I will explain that in detail here.  Below are the steps that need to be taken:
  • Create a cube with 127 dimensions
To help you create a cube with 127 dimensions, I have created a TI process.  At this time of writing I am not sure, how to attach a file (other than media) to the blog.  Therefore I am pasting the code below.  In this process, couple of parameters are defined and the code exists only in the Prolog tab.

vs_CubeName = pCubeName;
vi_MaxDims = 127;
IF (CubeExists (vs_CubeName) > 0);
    ItemReject ('Cube already exists. Quitting program with error');
    ProcessError;
ENDIF;
WHILE (vi_MaxDims > 0);
    vs_DimName = pDimName | NumberToStringEx (vi_MaxDims, '000', '', '');
    IF (DimensionExists (vs_DimName) = 0);
        DimensionCreate (vs_DimName);
        DimensionElementInsert (vs_DimName, '', vs_DimName, 'N');
    ENDIF;
    vi_MaxDims = vi_MaxDims - 1;
END;
CubeCreate (vs_CubeName,
    pDimName | '001', pDimName | '002', pDimName | '003', pDimName | '004', pDimName | '005', pDimName | '006', pDimName | '007', pDimName | '008', pDimName | '009', pDimName | '010', pDimName | '011', pDimName | '012', pDimName | '013', pDimName | '014', pDimName | '015', pDimName | '016', pDimName | '017', pDimName | '018', pDimName | '019', pDimName | '020', pDimName | '021', pDimName | '022', pDimName | '023', pDimName | '024', pDimName | '025', pDimName | '026', pDimName | '027', pDimName | '028', pDimName | '029', pDimName | '030', pDimName | '031', pDimName | '032', pDimName | '033', pDimName | '034', pDimName | '035', pDimName | '036', pDimName | '037', pDimName | '038', pDimName | '039', pDimName | '040', pDimName | '041', pDimName | '042', pDimName | '043', pDimName | '044', pDimName | '045', pDimName | '046', pDimName | '047', pDimName | '048', pDimName | '049', pDimName | '050', pDimName | '051', pDimName | '052', pDimName | '053', pDimName | '054', pDimName | '055', pDimName | '056', pDimName | '057
', pDimName | '058', pDimName | '059', pDimName | '060', pDimName | '061', pDimName | '062', pDimName | '063', pDimName | '064', pDimName | '065', pDimName | '066', pDimName | '067', pDimName | '068', pDimName | '069', pDimName | '070', pDimName | '071', pDimName | '072', pDimName | '073', pDimName | '074', pDimName | '075', pDimName | '076', pDimName | '077', pDimName | '078', pDimName | '079', pDimName | '080', pDimName | '081', pDimName | '082', pDimName | '083', pDimName | '084', pDimName | '085', pDimName | '086', pDimName | '087', pDimName | '088', pDimName | '089', pDimName | '090', pDimName | '091', pDimName | '092', pDimName | '093',
pDimName | '094', pDimName | '095', pDimName | '096', pDimName | '097', pDimName | '098', pDimName | '099', pDimName | '100', pDimName | '101', pDimName | '102', pDimName | '103', pDimName | '104', pDimName | '105', pDimName | '106', pDimName | '107', pDimName | '108', pDimName | '109', pDimName | '110', pDimName | '111', pDimName | '112', pDimName | '113', pDimName | '114', pDimName | '115', pDimName | '116', pDimName | '117', pDimName | '118', pDimName | '119', pDimName | '120', pDimName | '121', pDimName | '122', pDimName | '123', pDimName | '124', pDimName | '125', pDimName | '126', pDimName | '127');
ViewCreate (vs_CubeName, 'All');
  • Save and run the TI Process.  You will now see the cube name you supplied in parameter created and has 127 dimensions
  • Map the data source of the process to use a view on this 127 dimension cube
To do this, open the ExportToSPSS process in Architect.  Click Ok on any errors you get.  In DataSource Tab, click on Browse and select the cube you created and choose "All" as the view

After this, click on the Variables tab, you will be prompted with couple of options.  Choose the one highlighted

You can now go to the Data Tab, do the modification as recommended in the KB Article.  Comment out the old line and add the new line and save the process.  Ensure that preview in the IBM Cognos TM1 Import Palette runs correctly.
# vValue = if(VALUE_IS_STRING = 1, SVALUE, NumberToString(NVALUE));
vValue = if(VALUE_IS_STRING = 1, SVALUE, NumberToStringEx(NVALUE,'#####0.0##', '.', ','));

Closing Thoughts

  • If you are on IBM SPSS Modeler Ver 17, you will not face the issue of last digit being dropped
  • Modifying the code through TM1 Performance Modeler is fastest; but it increases the file size by 2.5 times.  We will need a file compare utility to figure out what has been additionally added by TM1 Performance Modeler
  • Using TM1 Architect is clean, however you will need to follow certain steps, before you are ready to make the change
  • In Ver 17, the file is named slightly different and the code difference between ver 16 and 17 is huge.  See below screen

  • As mentioned in my last blog, if you upgrade to a newer version of SPSS, there is need to copy these 3 files all over again to the various DATA directory of the TM1 server you are working with

Wednesday, June 17, 2015

IBM Cognos TM1 and Stored Procedures


We often use ODBC as Datasource Type in a Turbo Integrator (TI) process and load data into IBM Cognos TM1 cubes or build metadata.  In majority of the cases, we end up writing SQL code in Query box, to fetch the information that is required to perform the intended task.  Once in a while, we are required to work with a Stored Procedure (SP) in IBM Cognos TM1.

We will  look at how to execute an SP from TI. I will be using DB2' sample database to demonstrate the example.  SP name is TM1SP1 and its body is as follows.  A very simple SP, which accepts department number as a parameter, it then brings back the name and manager number associated with that department.

CREATE OR REPLACE PROCEDURE TM1.TM1SP1 (IN SrchNo char (3))
    DYNAMIC RESULT SETS 1
    LANGUAGE SQL
BEGIN
    DECLARE CUR1 CURSOR WITH RETURN TO CALLER FOR
        SELECT DeptNo, DeptName, MgrNo FROM TM1.DEPARTMENT WHERE DeptNo = SrchNo;
    OPEN CUR1;
END


Create a new TI process using ODBC as datasource type.  Choose the appropriate DSN, key in username and password.


For MS SQL if single-signon is enabled then, there is no need to key the username and password.  In our case, DB2 is not configured.  Therefore a named user and password is required to connect to the database.


 To run a stored procedure in DB2, you will need to use CALL function.  In our case, to run the TM1SP1 stored proc, the command will be 
In MS SQL or Oracle, you will use Execute SP_Name to run a SP.  Alternately you could abbreviate the Execute command to Exec and it will still work.

The preview pane of the TI task will now show the output of the stored procedure.  The test SP was written to return 3 columns from the Department table.  Preview pane in the TI window will show these 3 columns.

You can now switch to the Variable tab in TI window and work forward from there.

So far so good. This method will work.  As long as the stored procedure does NOT modify the data (i.e. there are no insert/update/delete statements in it).

Let's move on to Employee table in our sample database and look at Salary of Ted

SELECT EmpNo, FirstNme, LastName, Salary FROM TM1.Employee where EmpNo = '000100'; 
We will write an SP that increments salary of an employee by 1,000

CREATE OR REPLACE PROCEDURE TM1.TM1SP2 (IN SrchNo char (6))
    DYNAMIC RESULT SETS 1
    LANGUAGE SQL
BEGIN
    UPDATE TM1.Employee
    SET Salary = Salary + 1000
    WHERE EmpNo = SrchNo;
END

Every time a call to the SP is made, it increments the salary of employee number (passed as parameter) by 1,000.  Going by the above steps, if we end up creating a SP and write our TI as below
When you hit preview pane, you will see the results as below:

Ted's salary is already incremented by 1,000 to 87,150.  Let's proceed to variables tab and define the available 4 variables as Other.  In data tab, let's put this statement

AsciiOutput (GetProcessErrorFileDirectory | 'sk.txt', EmpNo, FirstNme, LastName, NumbertoString (salary));


Save the process and run it.  After the process is run, open the text file - sk.txt in our case.  Salary of Ted is incremented by another 1,000 to 88,150!


While the intention was to run the SP once and increment the Salary for given employee, by 1,000.  Net effect is that Salary is now incremented by 2,000.  Another unintended consequence of this approach is that, if someone just opens this TI process in TM1 Architect, then also the Salary is incremented!

Therefore, if the SP we are working with modifies the data, then it is better to run it in the prolog by using TI function DatasourceQuery.  

In our example, while we want to run the Stored Procedure in Prolog of Cognos TM1's TI, in the initial query pane, there is a need to have a valid SQL.  Since the Stored Procedure returns 4 columns, we will write out a sql that gets 4 columns but does nothing.


 When you hit Preview this is what is seen
Prolog tab will have the following code.
vs_SQL = 'Call TM1.TM1SP2 (''000100'')';
DatasourceQuery = vs_SQL;

Let the AsciiOutput statement remain as-is in the Data tab.  Set Ted's Salary back to 86,150.  

From TM1 Architect, run the modified TI process.  If we check the output file as well as the DB2 Table contents, we will now see that Salary is incremented by 1,000 alone.  Even if someone repeatedly opens the TI, the Salary is not changed - unless the process is run.

Summary:

  • To run a stored procedure from a TI in IBM Cognos TM1, we can write the statement in the query pane.  This is ok as long as the SP fetches the data
  • If however, the SP we want to work with modifies the data in some fashion (be it insert, update, delete or some combination of these) then writing the statement in the query pane, will lead to double/triple/quadruple/multiple executions, causing lot of grief