Query Oracle from SQL Server

Start with the following steps:

• Install the Oracle Instant Client (OIC) on the machine where SQL Server is running and add the "Oracle Provider for OLE DB" driver, e.g. to folder c:\oracle11g.
• For a SQL Server x64 version running on a Windows x64 OS download the instantclient-basiclite-windows.x64-11.2.0.2.0.zip. This is a 22 MB download.
• Then add the Oracle Data Access components, to get the "OLE DB for Oracle" driver:  http://www.oracle.com/technetwork/database/windows/downloads/index-090165.html
  At the time of writing (March 2012): select the "64-bit ODAC 11.2 Release 4 (11.2.0.3.0) Xcopy for Windows x64". This is a 54 MB download
• Inside SQL Server Management Studio:
  Linked Servers -> Providers -> OraOLEDB.Oracle -> Allow inprocess = true
• Optional: to enable adhoc querying (querying without a linked server), then enable this:
• SQL Server 2005 via surface area configuration tool
• SQL Server 2008 via facets: inside SQL Server Management Studio, right click the server -> facets -> facet: surface area configuration -> AdHocRemoteQueries = true
• Optional: if the connection to Oracle should be based on a TNS names entry, then add an "admin" folder to the OIC folder. Put or create an tnsnames.ora file there, e.g. c:\oracle11g\admin\tnsnames.ora
• Optional: use "tnsping" to test the entry from the command prompt, e.g. "tnsping dwh.world". If it doesn't recognize the entry, then create an environment variable called "TNS_ADMIN" and let it point to this "admin" folder, e.g. c:\oracle11g\admin
• Optional: if the following steps to get data from an Oracle instance don't work, then edit this registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC\MTxOCI.
  Set the following values:
  OracleXaLib=oraclient11.dll
  OracleSqlLib=orasql11.dll
  OracleOciLib=oci.dll

Note: these DDLs can be found in the c:\oracle11g\product\11.2.0\client_1\BIN folder. The specific version depends on the Oracle Client and might be for example oraclient10.dll en orasql10.dll. Use the names of the DDLs as found in this folder.

Three ways to get the data from an Oracle instance

Example:

Server: locksnlatches
Oracle instance (SID): dwh
Port: 1521
Schema/user: scott
Password: tiger
tnsnames.ora entry:

dwh.world =
(DESCRIPTION=
  (CID=DWH_APP)
  (ADDRESS_LIST=
    (ADDRESS=
      (PROTOCOL=TCP)(HOST=locksnlatches)(PORT=1521)
     )
   )
  (CONNECT_DATA=
    (SID=dwh)
  )
)

1) Using a linked server

• Linked Servers -> New linked server... ->  General ->
• linked server: lnk_locksnlatches (this name can be anything)
• provider: Oracle provider for OLE DB
• product name: oracle (this name can be anything)
• data source: (DESCRIPTION=(CID=DWH_APP)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=locksnlatches)(PORT=1521)))(CONNECT_DATA=(SID=dwh)))
• provider string: leave empty!
• location: leave empty!
• catalog: leave empty!
• Linked Servers -> New linked server... ->  Security ->
• Be made using this security context
• Remote login: scott
• With password: tiger

Use the four-point notation to retrieve data from Oracle:

SELECT 1 FROM lnk_locksnlatches..scott.dual

Note: the "security" tab specifies the Oracle schema/user and password used by the connection!

2) Without a linked server (TNS-less)

SELECT * FROM OPENROWSET('OraOLEDB.Oracle', '(DESCRIPTION=(CID=DWH_APP)(ADDRESS_LIST= (ADDRESS=(PROTOCOL=TCP)(HOST=locksnlatches)(PORT=1521)))(CONNECT_DATA=(SID=dwh)))';'scott';'tiger','SELECT 1 FROM DUAL') x

3) Without a linked server (TNS-full)

SELECT * FROM OPENROWSET('OraOLEDB.Oracle', 'dwh.world';'scott';'tiger','SELECT 1 FROM DUAL') x

Power view on analysis services tabular

How to create a Power View report using Sharepoint 2010 and Analysis Services 2012 Tabular as the data source?

0) Prerequisites
A server called "bisql01" with the following services installed:

•   sql server 2012 (mssqlserver = default instance)
•   analysis services 2012 (mssqlserver = default instance)
•   analysis services 2012 tabular mode / vertipaq (astabular = named instance)
•   reporting services (mssqlserver = default instance)
•   sharepoint 2010 installed at https://powerview.locksnlatches.com

All services run under user "cloud\bi01_sql". Work is done locally on this server and refered to as "localhost". Data is loaded from the SQL Server database "CPDV" into the Analysis Services database "CPDM".

INSIDE VISUAL STUDIO 2012

1) create project
new project -> analysis services tabular project
  name: cp_windows7

2) set deploy target of model
rightclick project "cp_windows7" -> properties ->
  server: localhost\astabular
  database: cpdm
  impersonation settings: default

note: there is probably also a default instance of analysis services running; this is the regular multidimensional cube instance like the previous versions of sql server, not the named tabular model instance

note: the cpdm tabular database will be created on deploy

note: there will also be a work tabular database for the current project named after the project, first data source and a guid, e.g. CPDV_BI01_SQL_01e0810e-c8ab-46fd-afe6-098348336a9a. Do not delete this database from the analysis server server!

3) add tables to model
model -> import from datasource -> microsoft sql server:
  server name: bisql01 (or "localhost")
  log on to the server: use windows authentication
  database: cpdv
  use service account,
  select tables/views,
  finish

edit the bism model: model -> model view -> diagram view. If there are no relationships in the source, then they can be added here by dragging and dropping fields from a child to a parent table or vice versa.

change connection type: model -> existing connections -> edit
  use sql server authentication
  name: sa
  password: ***

note: use model -> existing connections, to add tables later on from the same connection

4) set date dimension
view the data: model -> model view -> data view

click the date column, table -> date -> mark as date table

note: mark as date table does not work in the diagram view, only in the data view

note: make sure the date column is unique and continuous

note: a fact table can join to the date dimension on this date field, but also on a unique number field. A date field also needs to be selected in the latter case for "mark as date table" to work.

5) set authorization
models -> roles -> new
  name: reader
  permissions: read

models -> roles -> members -> add
  cloud\bi01_sql
  cloud\bi01_admin (=sharepoint account from which powerview is started)

note: if members are added to this role from management studio to the tabular model instance database then they will be overwritten at the next deploy of the project! So, specify the members in the project.

6) build -> deploy

INSIDE SHAREPOINT 2010

7) create semantic model connection
documents -> new document -> bi semantic model connection ->
  filename: model.bism (this is the name of the document in sharepoint; it can be anything)
  servername: bisql01\astabular
  database: cpdm

note: to fix error "database cannot be found", double check that the reader role has read permissions and that the windows accounts used by Sharepoint are added to that group

8) create report data source to semantic model connection
documents -> new document -> report data source ->
  data source type: microsoft bi semantic model for power vier
  connection string: datasource='https://powerview. locksnlatches.com/Shared%20Documents/model.bism'
  credentials: stored credentials
    username: cloud\bi01_sql
    password: ***
  availability: enable

9) create powerview report
click dropdown arrow of report data source -> create power view report

note: to fix error "User 'DOMAIN\user' does not have required permissions": start IE in "run as administrator" mode and connect using "localhost" iso. the server name.

note: the reporting services logfiles are located here C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\LogFiles

Quipu connections

Several ways to create a database connection in the open source data warehouse generator Quipu.

SQL Express

• SQL Server configuration manager -> 
• Protocols for SQLExpress -> TCP/IP: enabled
• TCP/IP -> IP Adresses -> IP All -> TCP Dynamic Ports: empty
• TCP/IP -> IP Adresses -> IP All -> TCP Port: 666
• SQL Server management studio -> SQL Express server properties -> Security -> Server Authentication: SQL Server and Windows Authentication mode
• Create a SQL Server authentication login on the database server and assign it to the database that will be used by the connection, e.g. login: quipu, password: quipu
• If the database runs on the same machine as Quipu, use "localhost" as the host name. Otherwise use the remote host name

Note:

• Do not use the instance name, e.g. using "AdventureWorks;instance=sqlexpress" as database works, but will, depending on the SQL Server version, result in an "unable to get information" error.

Combine two DVD's into one

When you want to combine two DVD's onto one disc, but keep the chapters, then try the following steps. I assume the video_ts folders of both DVD's are already available on your harddrive.

In most cases there is no vob file per chapter, but each vob file is filled to its maximum (1GB) and then a new one is created. Sometimes, however, there is a vob file per chapter. In this case, you only need to follow the steps following DVDStyler:

• Download and install DVDShrink (freeware)
• Start DVDShrink and click "Reauthor"
• In the right pane under "dvd browser", navigate to the video_ts folder of the first DVD
• Drag the desired chapters to the left
• In the right pane under "dvd browser", navigate to the video_ts folder of the second DVD
• Drag the desired chapters to the left
• Under File, click "Backup" and save to folder c:\shrink. This set automagically creates a vob file per chapter!
• Download and install DVDStyler (freeware)
• Start DVDStyler and add all .vob files from c:\shrink to the current project. The DVD main- and chapter menu's are created automatically 
• Optionally: add the names of the chapters to the chapter menus
• Burn the project to an iso file or directly to disc

Ofcourse, you can only do this with homemade DVD's or to DVD's you own legally and when local law permits it.

Use 32-bits Excel Source with 64-bits SSIS

The following is still work in progress and more of a braindump than anything else, but I've put it here, because someone might already find some use for it

Straight out of the box, you cannot use the Excel Datasource in SSIS on a 64-bits Windows platform , because the Excel Datasource uses a 32-bits OLEDB library. Microsoft has not released a 64-bits version of this library.

0) Use MADE 2010
There is 64 bits version for Office 2010, so for Excel 2010. It is called "Microsoft Access Database Engine  2010 Redistributable" of which there is a 32- and 64-bits version. We're interested in the 64-bits version, but this package can only be installed after removing all 32 bits Office applications!

a) Install MADE 2010 while NOT removing 32 bits Office applications:

AccessDatabaseEngine_x64.exe /passive

Notes:

• THE SYSTEM WILL AUTOMATICALLY REBOOT, SO SAVE WORK BEFOREHAND!
• Start Excel once to go through the configuration process. Start it a second time. It the configuration process starts again, then disable it by renaming this file to "setup_disabled.exe":
  c:\Program Files (x86)\Common Files\microsoft shared\OFFICE14\Office Setup Controller\setup.exe
• run the repair option for the Microsoft Access database engine 2010 program inside "add/remove programs" 

b) If that doesn't work, then also install the Data Connectivity Components:
http://www.microsoft.com/en-us/download/confirmation.aspx?id=23734

1) Trick Windows to use 32-bits ODBC data sources
Try adding the SysWow64 folder to the beginning of your path:

Path=%systemroot%\SysWow64;...old values here...

When done via the command prompt via the SET-command, it will be undone when that window is closed.

2) Run SSIS in 32-bits mode
Use the 32-bit DTEXEC.EXE to execute the package containing the Excel Datasource. It is located in "Program Files (x86)\Microsoft Sql Server\90\Dts\Binn" (or for SSIS 2008: "Program Files (x86)\Microsoft Sql Server\10\Dts\Binn")

You can schedule this using "task scheduler" or SQL Agent.

3) Compile package as 32-bits
Package configuration properties -> Debugging -> Debugging Options -> Run64BtRuntime -> set to False.

4) Even dirtier trick: convert Excel spreadsheet to CSV beforehand
This can be done in several ways, with minimal coding required:

• If Excel is installed, then create a VBS script that uses CreateObject("Excel.Application") to create an Excel object. Then open the XLS file and save it as a CSV using this object.
• ... or a perl script using the "Spreadsheet::ParseExcel" module
• ... or a powershell script using "comobject Excel.Application"
• Install OpenOffice and program a small macro in it's OpenOffice basic language. Then create a VBS or DOS script that opens OpenOffice and calls the macro with parameters, one being the source XLS, and one being the target CSV.
• Without installing anything on the server running SSIS: write a small C# application that utilizes the Excel OLEDB data provider built into Windows to read data from the Excel sheet. Then write the results to a CSV file.
• ... or write a small C# application that utilizes the opensource Excel Data Reader library to read data from the Excel sheet. Then write the results to a CSV file. This library appears to be buggy however.
• Freeware commandline tool: XLS2CSV by Zoom Technology. Parses only the first sheet. The column separator and text quotes are configurable throught an ini file
• Freeware commandline tool: XLS2CSV by Tom Crow. Parses all sheets to separate CSV files

ODBC/DSN configuration can be found here:

• 32 bits ODBC connections: %windir%\syswow64\odbcad32.exe
• 64 bits ODBC connections: %windir%\system32\odbcad32.exe

Notes:

• ADO also uses the OLEDB library.
• The OLEDB library is a piece of code that gets loaded and compiled into the calling application at runtime.
• If you use the MADE 2010 Redistributable package, then be aware of a small bug when connecting to Office 2010. You have to use "MICROSOFT.ACE.OLEDB.12.0" as the driver name in stead of the specified "MICROSOFT.ACE.OLEDB.14.0".
• Although previous Office versions need to be de-installed before installing MADE 2010, they can be reinstalled afterwards.

SQL Server statistics

Statistics are information about the distribution of data in a column, maintained internally by the database engine.

Rules and behaviour with regard to statistics, when automatic creation/updating is turned on:

• Statistics are automatically created when creating an index, but only for the first column of the (composite) index.
• Statistics can be created manually on any column or combination of columns.
• Statistics are automatically created/updated when the query optimizer finds need for them.
• Statistics are not automatically created/updated when the server is too busy, unless the "update statistics asynchronously" option is turned on, because then the are created in parallel.
• Asynchronously created/updated statistics have effect the next time the (causing) query is run.

Links:
http://www.sql-server-performance.com/tips/update_statistics_p1.aspx
http://blogs.technet.com/b/rob/archive/2008/05/16/sql-server-statistics.aspx
http://technet.microsoft.com/nl-nl/library/cc966419(en-us).aspx