External tables

Data can be imported into and exported from Oracle 10g in different ways. E.g. by using the SQL*Loader tool for text files (sqlldr) or the Data Pump (impdp/expdp) tools for the propriety Oracle RAW format. For previous Oracle versions, the tools imp and are also available. Those import/export to external locations on the client machine, not the server machine.

Another way to get data in and out of Oracle is to define an external table. To do this you have to create a table in Oracle in the regular way using a CREATE TABLE statement, but with adding an additional ORGANIZATION EXTERNAL clause.

Within this clause the properties of the external (server side!) location can be specified.

EXAMPLE

CREATE TABLE TEST
(
  "CODE" VARCHAR2(17 BYTE)
)
ORGANIZATION EXTERNAL
(
  TYPE ORACLE_LOADER
  DEFAULT DIRECTORY DATA_PUMP_DIR
  ACCESS PARAMETERS (
    RECORDS DELIMITED BY NEWLINE
    CHARACTERSET WE8MSWIN1252
    FIELDS TERMINATED BY 0X'09'
    MISSING FIELD VALUES ARE NULL
    REJECT ROWS WITH ALL NULL FIELDS
  )
  LOCATION ( DATA_PUMP_DIR: 'EXTFILE.TXT' )
)

FIELD LIST

The list of fields following the create table keywords cannot be inferred dynamically, but has to be specified explicitly. So it is not possible to "hook up" a text file and retrieving the list of fields automatically.

Trick: import the data first into a table using SQL Developer, TOAD, etc, and create the DDL script from it. Finally, copy and paste the field list from this script.

It is also possible to specify the "external" list of fields inside the organization external clause. This is, for example, similar to how it is done using Oracle Warehouse Builder (OWB).

TYPE

One of these properties is "TYPE", which specifies the access driver that Oracle uses. There are two values for this property:

• ORACLE_LOADER: only read access, from all kinds of text files (but not, for example, Excel .xls)
• ORACLE_DATAPUMP: read and write access, a propriety RAW format

The access driver is a wrapper Oracle uses to call the previously mentioned SQL*Loader or Data Pump functionality.

An error occurs when an insert statement is performed on a table is created with access driver ORACLE_LOADER:

SQL Error: ORA-30657: operation not supported on external organized table30657.0000 -  "operation not supported on external organized table"
*Cause:    User attempted on operation on an external table which is
           not supported.
*Action:   Don't do that!

The ORACLE_DATAPUMP type has to be used to export data from Oracle using an external table. Unfortunately, this data cannot be humanly read or imported into any other tool then an Oracle database.

DEFAULT DIRECTORY / LOCATION

A directory object has to be available in Oracle. If it is not already present, it has to be created. For example:

CREATE OR REPLACE DIRECTORY DATA_PUMP_DIR AS 'C:\'

Now the tag DATA_PUMP_DIR can be used in the organization external clause. The DEFAULT DICTIONARY property is mandatory and has to be used when setting the LOCATION.

A path specification is not allowed between the single-quotes.

This is not valid:

• LOCATION ( 'C:\EXTFILE.TXT' ) 

ACCESS PARAMETERS

The options are rather straightforward, except perhaps "FIELDS TERMINATED BY". To set the field delimiter a character can be specified between single-quotes. But it is also possible to specify special characters like tab or linefeed.

To specify a tab:

FIELDS TERMINATED BY 0X'09'

The 0X instructs Oracle that the value between the single-quotes should be interpreted as an ASCII value. In the example, ASCII character 9 is a tab.

OPTIONALLY ENCLOSED BY '"'

It is also possible to specify that some values have double-quotes around them. This however does not work in the following case:

Column 1 value - tab - tab - Column 3 value

Here a record has a value for columns 1 and 3, but not for column 2. It should be read as NULL, but is actually skipped. This results in the value of column 3 being wrongly inserted in column 2.

CHARACTER SET

Explicitly setting the character set of the external location prevents problems with interpreting the data and can improve performance.

From Odi's blog:

If you do not specify the CHARACTERSET parameter, the database character set is used to interpret the input file. As the input file is in WE8MSWIN1252, the ORACLE_LOADER driver gets confused trying to interpret single-byte WE8MSWIN1252 codes as multibyte AL32UTF8 codes.

The character set of the input file depends on the way it was created. Even on US Windows, you can create text files in different encodings. Notepad allows you to save the file in ANSI code page (=WE8MSWIN1252 on US Windows), Unicode (=AL16UTF16LE), Unicode big endian (=AL16UTF16), and UTF-8 (=AL32UTF8). The Command Prompt edit.exe editor saves the files in the OEM code page (=US8PC437 on US Windows).

-- Sergiusz

LOGGING

All errors are logged to a bad file - containing the violating records - and a log file - containing messages about the error that occured.

To disable logging, (some of) the following keywords can be added to the clause:

• NODISCARDFILE
• NOBADFILE
• NOLOGFILE

Oracle specification of the ORACLE_LOADER access driver:

http://www.stanford.edu/dept/itss/docs/oracle/10g/server.101/b10825/et_params.htm

SQLite enforces foreign key constraints

As of version 3.6.19, SQLite finally supports foreign key constraints that can be enforced.

http://sqlite.org/releaselog/3_6_19.html

A pragma statement has to be execute on the database to enable the enforcement:

PRAGMA foreign_keys = true;

Measuring disk I/O speed

Disk thruput tester is a handy tool to test the sequential read and write performance of storage media, like harddisks and SSD. It supports configurating custom block sizes:

http://www.objectso.nl/

Using this tool, testing the read/write performance of 8KB, 16KB, 32KB, etc, sized blocks is a breeze. The size and location of the test file is also configurable.

Note: on the website the tool is called "disk throughput tester".

Winload.exe missing or corrupt (0xc000000e)

Today we had a problem with Acronis Disk Doctor. We used it to try to resize a existing partition. The tool doesn't directly support dynamic disks - only after downloading additional drivers. These were downloaded, because we had those in our system, and the system had to be rebooted. No changes had been made to this point. But after the reboot, the system stopped almost immediately with a "black screen" reporting:

0xC000000E \WINDOWS\SYSTEM32\WINLOAD.EXE INFO: THE SELECTED ENTRY COULD NOT BE LOADED BECAUSE THE APPLICATION IS MISSING OR CORRUPT

What made this case different from all others that I could find on the internet is that the system partition was no longer accessible. It was not assigned the C drive letter.

So, the suggestions I could find did not work. They include, but are not limited to:

• Repair using the Vista installation disc
• From the Vista installation disc's command prompt option:
  bootrec /fixmbr -> completes succesfully, but didn't fix the issue
• From the Vista installation disc's command prompt option:
  bootrec /fixboot -> no system partition can be found
• From the Vista installation disc's command prompt option:
  bootrec /scanos -> no system partition can be found
• Perform the following three commands from the Vista installation disc's command prompt option:
  x:\windows\system32\bcdedit /set {default} device partition=c:
  x:\windows\system32\bcdedit /set {default} osdevice partition=c:
  x:\windows\system32\bcdedit /set {bootmgr} device partition=c:
  -> no system partition can be found

Bootrec is the "fdisk /mbr" for Windows Vista and Windows 7.

If you got here and you still have a C drive letter assigned and that drive is accessible, then the suggestions above might work. I suggest you try them first.

If you don't have a system drive letter, then the resolution is fairly simple:

Start the repair option from a Windows 7 installation disk.

It detected that there were issues with the current Vista installation and fixed them. We rebooted the system and all was well again. No data loss and still running Vista.

We used a Windows 7 Ultimate x64 disk to fix a Windows Vista Ultimate x32 OS.

Cursor variants in Oracle 10g

Examples of different types of cursors. Each has its own pros and cons. Example 1 is the most straightforward and easiest to understand. The variants in example 3 perform better. I've put them here for reference.

-- 1) implicit cursor

DECLARE l_old_tbs VARCHAR(100) := 'USERS';
BEGIN
FOR item IN (
SELECT OWNER, TABLE_NAME FROM ALL_TABLES WHERE TABLESPACE_NAME = l_old_tbs )
LOOP
dbms_output.put_line(item.TABLE_NAME);
END LOOP;
END;

-- 2) explicit cursor

DECLARE l_old_tbs VARCHAR(100);
CURSOR c1 IS (SELECT OWNER, TABLE_NAME FROM ALL_TABLES WHERE TABLESPACE_NAME = l_old_tbs);
BEGIN
l_old_tbs := 'USERS'; -- set variable here, otherwise it is NULL!
FOR item IN c1 LOOP
dbms_output.put_line(item.TABLE_NAME);
END LOOP;
END;

-- 3a) local collection variable with multiple fields using BULK COLLECT and FORALL

CREATE TABLE TEST1 (OWNER VARCHAR(30), TABLE_NAME VARCHAR(30));
DECLARE l_old_tbs VARCHAR(100);
TYPE r_tmp IS RECORD (OWNER ALL_TABLES.OWNER%TYPE, TABLE_NAME ALL_TABLES.TABLE_NAME%TYPE);
TYPE t_tmp IS TABLE OF r_tmp;
tmp t_tmp;
BEGIN l_old_tbs:= 'USERS';
-- fill local collection "tmp" with one bulk operation
SELECT OWNER, TABLE_NAME BULK COLLECT INTO tmp FROM ALL_TABLES WHERE TABLESPACE_NAME = l_old_tbs;
-- update/insert/delete with one bulk operation; select not permitted.
-- no "loop" keyword, because following statement is performed as one batch. Check dbms_output, this is only one line.
-- always use "commit" statement
FORALL i IN tmp.FIRST..tmp.LAST
INSERT INTO TEST1 VALUES tmp(i);
dbms_output.put_line('first: 'tmp.FIRST', last: 'tmp.LAST);
COMMIT;
END;

-- 3b) local collection variable with single field using BULK COLLECT and FORALL

DECLARE l_old_tbs VARCHAR(100);
TYPE t_tmp IS TABLE OF ALL_TABLES.TABLE_NAME%TYPE;
tmp t_tmp;
BEGIN
l_old_tbs:= 'USERS';
-- fill "tmp" with one bulk operation
SELECT TABLE_NAME BULK COLLECT INTO tmp FROM ALL_TABLES WHERE TABLESPACE_NAME = l_old_tbs;
-- update/insert/delete with one bulk operation; select not permitted.
-- no "loop" keyword, because following statement is performed as one batch.
-- always use "commit" statement
FORALL i IN tmp.FIRST..tmp.LAST
INSERT INTO TEST1 (TABLE_NAME) SELECT tmp(i) FROM DUAL;
COMMIT;
END;

-- 3c) local collection variable with multiple fields using BULK COLLECT and FOR

SET SERVEROUTPUT ON;
SET SERVEROUTPUT ON SIZE UNLIMITED;
DECLARE
  l_old_tbs VARCHAR2(100);
  TYPE r_tmp IS RECORD (OWNER ALL_TABLES.OWNER%TYPE, TABLE_NAME ALL_TABLES.TABLE_NAME%TYPE);
  TYPE t_tmp IS TABLE OF r_tmp;
  tmp t_tmp;
BEGIN
  l_old_tbs:= 'USERS';
  -- fill local collection "tmp" with one bulk operation
  SELECT OWNER, TABLE_NAME BULK COLLECT INTO tmp FROM ALL_TABLES WHERE TABLESPACE_NAME = l_old_tbs;
  -- update/insert/delete with one bulk operation; select not permitted.
  -- with "loop" statement. Check dbms_output, there are as much lines as tmp.LAST.
  -- always use "commit" statement
  FOR i IN 1..tmp.COUNT
  LOOP
    --INSERT INTO TEST1 VALUES tmp(i);
    dbms_output.put_line('tablename: '||tmp(i).TABLE_NAME||', last: '||tmp.LAST);
    COMMIT;
  END LOOP;
END;

-- 3d) local collection variable with single field using BULK COLLECT and FOR

SET SERVEROUTPUT ON;
SET SERVEROUTPUT ON SIZE UNLIMITED;
DECLARE
  i INT := 0;
  l_tabname VARCHAR2(100);
  TYPE r_tmp IS RECORD (TABLE_NAME USER_TABLES.TABLE_NAME%TYPE);
  TYPE t_tmp IS TABLE OF r_tmp;
  tmp t_tmp;
BEGIN
  -- fill local collection "tmp" with one bulk operation
  SELECT TABLE_NAME BULK COLLECT INTO tmp FROM USER_TABLES WHERE UPPER(TABLE_NAME) LIKE 'TTMP_%';
  -- update/insert/delete with one bulk operation; select not permitted.
  -- with "loop" statement. Check dbms_output, there are as much lines as tmp.LAST.
  -- always use "commit" statement
  FOR i IN 1..tmp.COUNT
    LOOP
      dbms_output.put_line(i);
      l_tabname := 'DROP TABLE ' || tmp(i).TABLE_NAME; 
      dbms_output.put_line(l_tabname);
      EXECUTE IMMEDIATE l_tabname;
      COMMIT;  
    END LOOP;
END;

-- 4) use a temp table

CREATE GLOBAL TEMPORARY TABLE tmp (OWNER VARCHAR(30), TABLE_NAME VARCHAR(30);

Create handwritten truetype font for free using fontcapture

Fontcapture (http://www.fontcapture.com/) is the only website I could find that offers a service to convert handwritten characters into a True Type (.ttf) font for free. It supplies you with a template to fill in each of the glyphs (characters) of the font. You can print this template, fill it by pen, and scan and upload it to the fontcapture website. The bitmaps are turned into vector paths in the background and the font is created almost instantly.

Another advantage is that if you own a tablet pc or a (Wacom) pentablet, you can use it to fill in the template digitally. No need to print and scan the template.

The mechanism can also be used to trace bitmaps, since the tool does not distinguish between characters or other designs. The result is a font with vectorized images.

Alternatives:

• Fontifier (http://www.fontifier.com/): almost identical to Fontcapture, but charges $9 per font
• Yourfonts (http://www.yourfonts.com/): almost identical to Fontcapture, but charges $9 per font
• Fontgrinder: (http://www.fontgrinder.com/): idem, but charges even more: $14.95
• Fontstruct (http://fontstruct.fontshop.com/): free, but doesn't support handwritten fonts. The website has an editor for constructing your own font from predefined building blocks. A sort of "lego-for-font" method.

UPDATE 20100106: Unfortunately, the site has been taken offline. The domain is now referring to yourfonts.com. No free alternative is available anymore.

UPDATE 20110727: There is a new website offering the same free funtionality: http://www.myscriptfont.com

Netsend

Windows Vista and Windos 7 are no longer shipped with the command line tool netsend. Fortunately, a freeware alternative is available at:

http://www.lantalk.net/netsend/

List of freeware tools

Explorer & OS

• Total Commander (shareware): http://www.ghisler.com/
• 7-Zip: http://www.7-zip.org/
• VirtualCloneDrive: http://www.slysoft.com/nl/virtual-clonedrive.html
• NTFS Undelete: http://www.ntfsundelete.com/
• CDBurnerXP: http://cdburnerxp.se/en/download
• Atto Disk Benchmark: http://www.attotech.com/products/product.php?sku=Disk_Benchmark

Editor

• Notepad2: http://www.flos-freeware.ch/notepad2.html
• ExifTool & ExifToolGUI: http://www.sno.phy.queensu.ca/~phil/exiftool

3D

• Now3d: http://www.now3d.it/Eng/
• Blender: http://www.blender.org/
• Sketchup: http://sketchup.google.com/

Graphics

• FastStone Image Viewer: http://www.faststone.org/
• Paint .Net: http://www.getpaint.net/
• IcoFX: http://www.icofx.ro/
• Screenhunter Free: http://wisdom-soft.com/products/screenhunter.htm
• Inkscape: http://www.inkscape.org/
• FontForge: http://fontforge.sourceforge.net/

Conversion

• Super: http://www.erightsoft.com/
• MediaCoder: http://mediacoder.sourceforge.net/download.htm
• PDFsam (split and merge): http://www.pdfsam.org/
• Bullzip PDF Printer: http://www.bullzip.com/products/pdf/info.php
• A-PDF extractor (PDF to text): http://www.a-pdf.com/text/index.htm

Media & audio/dsp

• Renoise: http://www.renoise.com/
• Zoom Player 5.02 (6+ is no longer free): http://download.cnet.com/Zoom-Player-Standard/3000-2139_4-10068896.html?tag=mncol
• Media Tagger (Ladislav Dufek): http://www.softpedia.com/get/Multimedia/Audio/Tag-Editors/Media-Tagger.shtml
• Subtitle Workshop: http://www.urusoft.net/home.php?lang=1
• Divx 7: http://www.divx.com/en/win
• Youwashock: http://youwashockvst.googlepages.com/
• Stomper: http://www.lysator.liu.se/~zap/stomper/index2.html
• MP3Val: http://mp3val.sourceforge.net
• T@B ZS4: http://www.zs4.net

Databases

• PostgreSQL (as a service): http:/www.postgresql.org
• SQL Server Compact 3.5 (in-process): http://www.microsoft.com/Sqlserver/2005/en/us/compact.aspx
• SQLite (in-process): http://www.sqlite.org/

Great directors and actors

Just a list for myself, because I keep forgetting names.

Directors

• Darren Aronofski: Pi, Requiem For A Dream, The Wrestler
• Christopher Nolan: Memento, Insomnia, Batman Begins, The Prestige, The Dark Knight
• Clint Eastwood: The Bridges Of Madison County, Mystic River, Million Dollar Baby, Letters From Iwo Jima, Changeling, Gran Torino
• David Fincher: Se7en, The Game, Fight Club, Zodiac, The Curious Case Of Benjamin Button

Actors

• Christian Bale: Empire Of The Sun, Equilibrium, The Machinist, Batman Begins, Harsh Times, The Prestige, 3:10 To Yuma, The Dark Knight
• Joseph Gordon-Levitt: Mysterious Skin, Brick, The Lookout

Oracle SQL Developer screen redraw glitch

Perform the following steps to fix the redraw, or screen refresh, bug that can occur in Oracle SQL Developer when used on Windows Vista and Windows 7. They occur for example when scrolling horizontally or vertically through the records of a table when the records don't fit the window.

Start with just the sqldeveloper.conf adjustment. If that doesn't fix it, then perform all steps.

Steps to solve the issue:

• Add or replace the attribute of the following config file.
  ..\sqldeveloper\bin\sqldeveloper.conf
  AddVMOption -Dsun.java2d.noddraw=true
• Install the most recent jdk.
  Last seen here: http://java.sun.com/javase/downloads/index.jsp
  Select the Java SE Development Kit (JDK). At this time, JDK 6 Update 16 is the most recent version. Install it in C:\Program Files\Java\jdk1.6.0_16
• Add or replace the attributes of the following config files.
  ..\ide\bin\jdk.conf
  SetJavaHome C:\Program Files\Java\jdk1.6.0_16
  ..\ide\bin\ide.conf
  AddVMOption -Xmx256M
  AddVMOption -Dsun.awt.keepWorkingSetOnMinimize=true

Remove visual rings / circles from pen tablet in Windows 7

When using a pen in stead of a mouse in Windows, rings or circles may appear when you click. This happens at least in Vista and Windows 7. I am using a Wacom Graphire pen and wanted to deactivate those. This is not as straightforward as it seems. The rings are part of the Tablet PC Input functionality of the OS - not of the Wacom drivers or application.

How to disable these rings in Windows 7?

• Disable the "Tablet PC Input Service".
  This solved the problem for me for a while... until the rings came back for no apparent reason. I suspect because of an automatic update or updated Wacom driver, that reactivated the tablet pc ring feature.
• Uninstall Tablet PC Components.
  Control panel -> Programs and Features -> Turn Windows features on or off -> uncheck Tablet PC Components. Windows needs to be restarted after this.
• Kill the wisptis.exe process, e.g. via taskmanager.
  Then 1) rename \windows\system32\wisptis.exe to wisptis.bak, 2) create a textfile called wisptis.exe with nothing in it. Alternatively: open properties for wisptis.exe, go to security tab and "deny full control" for the "everyone" account. If "everyone" is not visible, click edit, click add, type in "everyone" and click ok.

I am not using a tablet pc, so these actions might have a negative impact when using one.

UPDATE 20100305: The wisptis.exe process still comes up when running some .Net applications. Even when the above tip is applied. Most probably it is copied from the protected cache / version history Windows keeps. To prevent this, download the tool "process blocker":

http://www.processblocker.com/download.html

Configure it to block "wisptis.exe" and now the rings are REALLY gone forever.

UPDATE 20120718: There is a registry fix to disable the rings:

http://viziblr.com/news/2011/8/14/the-ultimate-guide-to-making-your-wacom-tablet-work-on-windo.html

Download the fix from step 5. Try this first and if it doesn't work, then use the process blocker application.

UPDATE 20130410: ... or disable it via a group policy:

1. Open Local Group Policy Editor:Run... gpedit.msc
2. Navigate to User Configuration - Administrative Templates - Windows Components - Tablet PC - Cursors
3. Enable the Turn off pen feedback setting.

Limited connectivity using wireless internet in Vista Home SP1

I've had an issue with "limited connectivity" on my Vista Home SP1 installation. It is a laptop that connects wirelessly (WiFi) to a router. The router is connected to the internet via DSL. The router is working fine and the connection to the internet is ok. However, after using the internet for some time, sometimes half an hour, sometimes minutes, the wireless connection displays a yellow exclamation mark with the message "limited connectivity". It is not the router, since rebooting the laptop fixes the issue... for some time, and then it happens again. After that, trying to open network connections, or any other application related to the network, hangs that application.

I've collected the following possible remedies. I'm not sure which one solved it in the end. Obvious remedies like "put the laptop wifi switch to ON", "put in a correct WEP key" (or any other encryption key) or "check if the router is turned on" are left out:

• (20090618) I put this upfront, because all the tips that follow didn't resolve the issue for me, so I finally bought a new Wifi USB dongle. After buying a new laptop to add to my wireless network I noticed that the WEP encryption I was using didn't work for the new laptop. So in the end I changed protocols from WEP to the stronger WPA. After that, all PC's and even the Wii worked flawlessly. Even after switching back to the original wifi NIC on the laptop that first had the "limited connectivity" message, that laptop hasn't had the problem. So in short: try changing encryption protocols. Start with none (unsecured) to see if the problem is gone and then move on to stronger ones.
• Update drivers. There is a great free tool to update almost all drivers on your system, called DriverMax. Just register and it can be used legitimately for 30 days. I am not sure why it is "freeware" if it states that you can "use it freely for 30 days". It is based around a community of users uploading drivers. It the tool detects a new driver, you can then update it. This is a bit tedious, but always better then scouring the internet manually. The current version of the driver that is going to be updated is uploaded to DriverMax first. It is stored in their driver base for other users.
• Reset TCPIP stack. Execute the following statement at the command prompt:
  NETSH INT IP RESET
• Reset Winsock. Execute the following statement at the command prompt:
  NETSH WINSOCK RESET. Or remove both winsock registry keys, reboot, and reinstall TCP/IP protocol: http://support.microsoft.com/kb/811259
• Disable auto-tuning. Execute the following statement at the command prompt:
  NETSH INTERFACE TCP SHOW GLOBAL. Check if "Receive Window Auto-Tuning Level" is disabled. If not, then disable it:
  NETSH INTERFACE TCP SET GLOBAL AUTOTUNINGLEVEL=DISABLED. Re-enable with "=NORMAL".
• Disable IP6 protocol. Uncheck IPv6 protocol on the network adapters and disable in the registry. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters, add or change DWORD key "DisabledComponents", set to hex "FF".
• Disable other protocols. Additionaly uncheck QoS Packet Scheduler, Link-Layer Topology Discovery Mapper I/O Driver, and Link-Layer Topology Discovery Responder.
• Disable power management on NIC. Right click the wireless network adapter in Device Manager, select properties, tab "Power Management" and uncheck the box "Allow the computer to turn of this device to save power". In my case it was turned back on again at a later point in time, but that could be due to installing a new driver afterwards.
• Change wireless channel. Try different channel numbers for the NIC. Each channel number (mostly between 1 and 13) corresponds to a different frequency. The property can be set on the advanced tab of the driver for the NIC in device manager. Set the adhoc channel accordingly in the wireless router configuration.
• Disable and re-enable network adapter. Apparently, this sometimes fixes issues.
• Reboot in Safe Mode. A solution like the previous one. Reboot the system, press F8, select "safe mode". After it has fully restarted, reboot again. Windows starts normally again.
• Run the System File Checker utility. Execute the following statement at the command prompt:
  SFC /SCANNOW. This is not of much of use to the majority of laptop owners, since they often do not own a copy of their Windows OS installation disk. When SFC finds a component (e.g. DLL) on the system, that is different from the one that was installed during the original installation, then it needs to get that original one from the installation disk.
• Check eventviewer. I have error messages concerning a NTIOMIN service that cannot start, TCPIP.SYS signing issues, and Kerberos related messages (can supposedly be ignored when the laptop is not part of a domain). Still haven't figured out the first two. I'm also getting "informational" events about the NETw5v32 wireless driver. This is related to an incorrect driver.
• Run registry or system cleaners: nice tools to run are HijackThis and CCleaner. The first can be used to remove questionable or corrupt services, browser helper objects, protocols, etc. The latter cleans the registry, removing orphanaged keys.
• Repair wireless. Right click the wireless network icon in the notification area and select "diagnose and repair". Alternative method. Execute the following statement at the command prompt:
  %windir%\system32\rundll32.exe ndfapi,NdfRunDllDiagnoseIncident. Do this before "limited connectivity" message pops up. In my case it displayed a message: "TCP/IP settings not optimized", and it optimized them. Running it again didn't display the message again, so I guess it did something the first time.
• Run Intel PROSet/Wireless diagnostics. If using Intel on-board wireless (I am using 3945ABG), then you can run the diagnostics utility. Execute the following statement at the command prompt:
  %windir%\system32\iPROSet.cpl. Run "diagnose" to check hardware, driver, radio, etc. Run "statistics" to check beacons, transmit errors, and power and power levels.
• Downgrade driver. Sometimes the most recent driver is not the best one for your device. Some drivers available are: NETw3v32, NETw4v32 (11.5.0.34), NETw5v32 (12.2.0.11). In my case, the NETw4v32 driver gave 100% signal quality, while a NETw3v32 driver gave only 60%. The NETw5v32 filled my eventlog with messages (see remark elsewhere).
• Set router to G-Only: put the router in network mode "G-Only". This way, the older B-mode is not supported, but throughput is improved.
• (Run Network Diagnostics tool. Execute the following statement at the command prompt: NETSH DIAG GUI. Check all scanning options and start. This should work in Vista when the command prompt is started with admin rights, but I couldn't get it to work)
• (NIC settings. If supported, try putting the adapter in full-duplex mode, or if it is already, in half-duplex mode. Experiment what works best. I cannot do this myself and wonder if this is even possible for a wireless adapter?)
• (Run netdiag tool. Unfortunately, this tool is only availabe in Windows XP as part of "Microsoft Windows XP Support Tools". So, only for those using Windows XP, execute the following statement at the command prompt:
  NETDIAG /TEST:WINSOCK /V. Use the outcome of the test for further analysis.)

Links: http://ask-leo.com/what_is_limited_connectivity_and_how_do_i_fix_it.html

Simple way to read Excel or Access data in SQL Server

-- list of available providers
exec master..xp_enum_oledb_providers

-- Three ways to read in Access data (file: c:\db1.mdb, tabel: TEST)

1a) Using openrowset with older MS Access version:
SELECT * FROM OPENROWSET('MSDASQL', 'Driver={Microsoft Access Driver (*.mdb)};Dbq=C:\db1.mdb;Uid=;pwd=;','SELECT * FROM TEST')

1b) For 2010 or more recent (*.accdb)
SELECT * FROM OPENROWSET('MSDASQL', 'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=C:\db1.mdb;Uid=;pwd=;','SELECT * FROM MAANDEN')

2) Using opendatasource:
SELECT * FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0','Data Source=C:\db1.mdb;')...[TEST]

3) Using a linked server called TEST for 2010 or more recent:

EXEC sp_addlinkedserver
@server= N'TEST',
@srvproduct = N'MSDASQL',
@provider= N'MSDASQL',
@provstr= N'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=C:\db1.mdb;Uid=;pwd=;'

-- Read in Excel (file: c:\employee.xls, worksheet: TEST)
SELECT * FROM OPENROWSET('MSDASQL', 'Driver={Microsoft Excel Driver (*.xls)};Dbq=C:\employee.xls;Uid=;pwd=;','SELECT * FROM [TEST$]')

-- Insert into text file
insert openrowset( 'Microsoft.ACE.OLEDB.12.0', 'Text;HDR=yes;Database=c:\tmp\', 'select * from test.csv') (testname, testaddress) select testname, testaddress from vw_test;

Note:

• MS Access needs to be installed on the same server where SQL Server is installed
• Make sure that the use of ad hoc names is allowed:
  exec sp_configure 'show advanced options', 1;
  RECONFIGURE;
  exec sp_configure 'Ad Hoc Distributed Queries', 1;
  RECONFIGURE;

Possible error messages and solutions:

OLE DB provider "Microsoft.ACE.OLEDB.12.0" for linked server "(null)" returned message "Unspecified error"

Give read/write access to "everyone" on the server to the following folder:

C:\Users\<SQL Server Service Account>\AppData\Local\Temp

Make sure this is set on the server where the code is running, i.e. on the client PC where the code is executed within Management Studio.

Converting WAV DTS to MP3

A WAV DTS file contains a multi-channel encoded digital format called DTS and has .wav as extention. Playing such a file in Zoomplayer or WinAmp typically produces a static sound. This is because a filter and/or decoder are not installed.

For Zoomplayer a filter can be installed (Gabest AC3), but a separate DTS decoder is still needed. Ffdshow contains such a decoder, but setting up this mapping in advanced options -> customized media playback -> source filters and splitters didn't help. Zoomplayer keeps displaying the "An ERROR has occured!" messagebox when trying to play the WAV DTS file.

Foobar2000 used to have an additional component, foo_dts.dll, that could be downloaded from the site, but it is no longer available. I was unable to find out why it is no longer supplied. After searching the internet I found a version of the library, but foobar2000 0.9.5 refused to load it, reporting that it was an outdated version.

A player that can play them out-of-the-box is VLC player. This player has a built-in filter and decoder to play WAV DTS files. It can also transcode files, e.g. to MP3 format, but when I use it on a WAV DTS file, the program terminates unexpectedly.

So, in the end, I was unable to find a "one-click" solution. Here is how I finally managed to convert a WAV DTS file into a MP3 file:

For the example, I use a WAV DTS file named c:\wavdts.wav

1. Download BeSplit (http://www.afterdawn.com/software/audio_software/audio_tools/besplit.cfm). I used version 0.9b6.2.
2. Extract the zip file to c:\
3. Start BeSliced.exe
4. Drag and drop wavdts.wav onto the besliced canvas
5. Select "DTS" when asked to select a format
   BeSplit is now ripping the DTS stream from the WAV DTS file.
   To use BeSplit without BeSliced, execute this command at the command prompt:
   BeSplit.exe -core( -input c:\wavdts.wav -fix -logfile C:\BeSliced.txt -type dtswav -output c:\wavdts_Fixed.dts ) -profile( BeSliced v0.3 )
   A file called c:\wavdts_Fixed.dts is created.
6. Download DTSDec (http://www.rarewares.org/others.php). I used version 0.0.2.7.
7. Extract the zip file to c:\
8. Execute this command at the command prompt:
   dtsdec -o wav > regularwav.wav wavdts_Fixed.dts.
   DTSDec is now transcoding multi-channel DTS to a regular two-channel WAV file called regularwav.wav
9. Use your favorite MP3 converting tools, e.g. AudioGrabber with LAME, to convert this WAV file to a MP3 file.

Money pit on Oak Island

Very bizarre story:
http://www.activemind.com/Mysterious/Topics/OakIsland/

I've put it here so I don't forget to check progress on the digging from time to time.

Update: an anonymous poster pointed me to the site http://www.oakislandtreasure.co.uk/. Check it out, as there are updates to this story!

Drupal Clean URL's with IIS7

Drupal can work with clean URL's (or URL write). The most talked about solution is using Helicon's ISAPI _Rewrite ISAPI add-in, which can be found here (http://www.isapirewrite.com/) if you are interested. But here I want to show you how you can accomplish clean url's using a free alternative by Ionics called IIRF.

Perform the following steps to get clean URL's in Drupal 5.3 in IIS7:

1) Download Ionics Isapi Rewrite Filter
At the moment of writing this is the only quite reliable freeware URL rewriter for IIS. You can get it here:
http://www.codeplex.com/IIRF/Release/ProjectReleases.aspx?ReleaseId=5018
Download the most recent "-bin.zip" version.

2) Install Ionics Isapi Rewrite Filter
- Extract IsapiRewrite4.dll to c:\inetpub\wwwroot (or your www root folder)
- Extract \examples\DrupalRules.ini to c:\inetpub\wwwroot (or your www root folder) as IsapiRewrite4.ini

If you haven't installed Drupal in the webroot, but, like me, in a subfolder, then you have to perform the following updates to DrupalRules.ini. I use http://localhost/drupal, so I had to add
add "/drupal" to almost every RewriteRule. In stead of "/drupal" you can use any folder or level of subfolders that is necessary in your case, e.g. for http://localhost/davy/dozy/tich you have to replace the bold "/drupal" string with "/davy/dozy/tich".

Here are some examples for each of the sections defined in DrupalRules.ini:

RewriteRule ^/misc/(.*)$ /misc/$1 [I,L]
RewriteRule ^/modules/tinymce/(.*)$ /modules/tinymce/$1 [I,L]
RewriteRule ^/cron\.php$ /cron.php [I,L]
RewriteRule /(.*)\?(.*)$ /index.php\?q=$1&$2 [I,L]
RewriteRule ^/(.*)$ /index.php?q=$1 [I,L]

become

RewriteRule ^/drupal/misc/(.*)$ /drupal/misc/$1 [I,L]
RewriteRule ^/drupal/modules/tinymce/(.*)$ /drupal/modules/tinymce/$1 [I,L]
RewriteRule ^/drupal/cron\.php$ /drupal/cron.php [I,L]
RewriteRule /drupal/(.*)\?(.*)$ /drupal/index.php\?q=$1&$2 [I,L]
RewriteRule ^/drupal/(.*)$ /drupal/index.php?q=$1 [I,L]

The following RewriteRule has to be added to the first RewriteRule-section If you also use Gallery2:

RewriteRule ^/gallery2/(.*)$ /gallery2/$1 [I,L]

or if you use http://localhost/drupal

RewriteRule ^/drupal/gallery2/(.*)$ /drupal/gallery2/$1 [I,L]

The RewriteRule function uses regular expressions to transform clean url's back to their non-clean equivalents.

3) Add ISAPI filter to IIS7
- Start IIS.msc
- Select Default Web Site (not your virtual directory or application!)
- Double-click the ISAPI filters icon
- Add...

Filtername: IsapiRewrite4
Executable: C:\inetpub\wwwroot\IsapiRewrite4.dll

4) Set Drupal to use Clean URL's
- Start Drupal in your web browser
- Administer -> Clean URLs
- Click "Test for Clean URL's" (it is somewhat hidden in the paragraph text)
- Select enable

In my case, the enable check box was still greyed out. To set Clean URL's manually:

- Remove write-protection from the c:\inetpub\wwwroot\drupal\sites\default\settings.php file
- Edit the file and add:

# Force Clean urls
$conf['clean_url']=1;

- Set write-protection back.

Hopefully this quick and rudimentary instruction helps you setting up URL rewriting for you Drupal installation.

Liger

I saw a documentary today on National Geographic about the cross-breeding product of a tiger and lion. The hybrid is called a liger. I didn't know that such a beast exists.

You can read more about it here:
http://en.wikipedia.org/wiki/Liger

Fix upload of images in Drupal

Solution to messages like

warning: move_uploaded_file() [function.move-uploaded-file]: Unable to move 'c:\progra~1\php\upload_tmp\php525F.tmp' to '' in C:\inetpub\wwwroot\drupal\includes\file.inc on line 244."

when trying to upload images to Drupal 5.3.

Perform or check these steps (I'm using Vista and IIS7):

1) Create temporary upload folder for php
php must have an upload folder to be able to upload files. Create one called upload_tmp under the install root of php. So if php is installed here:

c:\progra~1\php\

then the full path to the upload folder will be:

c:\progra~1\php\upload_tmp

2) Change php.ini
Change the settings to the following values:
file_uploads = On
upload_tmp_dir = "c:\progra~1\php\upload_tmp"
upload_max_filesize = 2M

3) Create temporary preview and upload folder for Drupal
Drupal must also have an upload and preview folder to be able to upload and preview files.

Create the following folders under the drupal root.
files
files\tmp

So if drupal is installed here:

C:\inetpub\wwwroot\drupal\

then the full path to the upload and preview folders will be:

C:\inetpub\wwwroot\drupal\files
C:\inetpub\wwwroot\drupal\files\tmp

In my case it worked without setting permissions on these folders, but if it doesn't, you can try giving the web service user access.

Give user IUSR permission to read/write on these folders:

- Right click folder- Properties -> security -> edit -> add -> advanced -> find now
- Select IUSR, ok
- Select full control, ok, ok

4a) Change drupal settings (upload module)
Activate the upload module:

In Administer -> Site building -> Modules, activate module Upload

4b) Change drupal settings (folders)
In Administer -> Site configuration -> File system, set the following values:

File system path: files
Temporary directory: files/tmp

The temporary directory must be a subfolder of the file system path folder!

Now the upload should work.

Install Windows Components from command-line

If you want to start the add/remove Windows Components dialog without using the add/remove programs interface, then use the following command:

Sysocmgr.exe /i:sysoc.ini

IE7 enable tabbed browsing

You can enable tabbed browsing in Internet Explorer 7 by using a menu command:

Tools -> Internet Options -> Tab: settings -> select Enable Tabbed Browsing.

However, sometimes this is not enough, for example because the domain administrator has "disabled" tabbed browsing using a registry key. You can enable it by putting the next registry entry in a .reg text file (e.g. c:\tabbrowse.reg) and executing it:

Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer]"ClassicShell"=dword:00000000

... or you can set the DWord keyvalue to 0 by hand.

If the key value gets restored every time you log in to your system, you could add the execution of registry file to the startup folder:

- create a bat file, e.g. tabbrowse.bat
- add the following line to the bat file: regedit /s c:\tabbrowse.reg
- add the bat file to the startup folder