Tuesday, November 10, 2009

Open Source Data Profilers

I've been looking into Open Source Data Profilers of late.
These allow you to get some basic info on your data quickly.
The 2 I like the most so far are the Talend Open Profiler
and another called Data Cleaner. Talend is more graphical and
Data Cleaner is a bit more intuitive so they both have their merits.

http://www.talend.com
http://datacleaner.eobjects.org

Talend also sells an "Enterprise" version of the product

I also came across an Open Source Data Modeling tool that has some
basic profiling capabilities named Power Architect.

http://www.sqlpower.ca

Tuesday, August 18, 2009

wwv_flow_epg_include_mod_local

At some point between Apex 2 and 3 Oracle decided tighten the security on calling stored procedures. I had a procedure that was being used to download documents from the database to the browser. This worked fine in Apex 2, but was broken after the upgrade. Upon clicking on the document link I was receiving a "Forbidden" message. This indicated a security problem. After some searching I came across others talking about a function that effectively allows you to "register" procedures that you would like allow to be called via the PL/SQL gateway. This procedure is in the APEX schema and is named wwv_flow_epg_include_mod_local.
I needed to comment out the return false at the beginning of the function and add the name of the procedure I wanted to allow calling to the list.
i.e.

if upper(procedure_name) in (
'QUANT.APEX_DOWNLOAD_NOTEBOOK_ENTRY') then
return TRUE;
else
return FALSE;
end if;

Thursday, August 13, 2009

RMAN switch to copy

Today I successfully used RMAN switch to copy to swap out my 500GB drives for a 1.5TB drives with minimal downtime. I have a simple setup where data resides on an E: drive and backups on the F: drive. The process was relatively simple.

1) Within RMAN execute backup as copy database
2) Shutdown the database
3) startup mount
4) Within RMAN execute switch database to copy
5) Within RMAN execute recover database
6) alter database open

At this point I had the database running from the F: drive with the exeption of the redo and control files. In my case these reside on C: which was unaffected by this execise, but something you should be aware of. Also, the Temp tablespace files was not moved from this. There may be an option to include the TEMP tablespace, however it was easy enough for me to just create a new default tempory TEMP tablespace (named FTEMP) and drop the old one.

Now the move back. This really was just the same exercise in the other direction. In order to make it work I needed to first change the db_recovery_file_dest to point back to the E: drive

1) alter system set db_recovery_file_dest=E:\oradata
2) Within RMAN execute backup as copy database
3) Shutdown the database
4) startup mount
5) Within RMAN execute switch database to copy
6) Within RMAN execute recover database
7) alter database open
8) Create a new default TEMP tablespace on E:
9) alter system set db_recovery_file_dest=F:\flash_recovery_area scope=both

At this point the database is back on the E: drive. However, we are vulnerable at this point so I immediately kicked off a database backup to get things back to a suitable state.

One side effect of this execrise was that my datafiles were renamed. Before the move I had a naming convention for datafiles using the tablespace_name_.DBF. For example,
for the QUANT tablespace QUANT_01.DBF, QUANT_02,DBF, etc... I don't know of a way to name datafiles according to this format using backup as copy database. You can use a custom format at the individual copy datafile level, but that seemed error prone since I would need to type in the file names by hand and run alot more switch commands.

Friday, July 17, 2009

Save Outlook attachements to disk

This handy bit of Powerscript code allows you to save attachements in your Outlook inbox to disk:

You need to pass a file name, not just a destination directory to SaveAsFile.The attachment has a FileName property.
This code groups attachments with same name and saves the first attachment in each group, if the attachment is corrupt or if it can't save it the Trap handles the exception; in this sample it just writes a message.You could also iterate through each group's attachments and append a different number to the name in order to save all.

$inbox=6
$outlook = new-object -com Outlook.Application
$inbox = $outlook.Session.GetDefaultFolder($inbox)
foreach ($group in $inbox.items % {$_.attachments} group filename) {
trap {
Write-Host There was a problem saving $fName
continue}
if ($group.Name.startswith("SECTSPDR")) {
$fName = "C:\TEMP\$($group.Name)"
$group.Group[0].saveasfile($fName)
if ($?) {Write-Host $fName was saved succesfuly.}
}
}

Backup options on your home network

I think I have settled on a good backup option for my desktop at home. I have an external hard drive connected to my desktop. Previously, I had installed rsync via cygwin which I have successfully used in the past. However, every few days I would find the process hung and had to reboot the machine. It appeared that this was not a common problem and it's not very easy finding support for free tools like cygwin (understandably)

Recently I cam across a tool similar to rsync from Microsoft named robocopy. This utility is included in the Windows resource tookit. It seems to be working well and have not yet found it hung like rsync was doing.

I also added an email to the job to mail the results of the run. For this I installed blat. I also had to hook into an SMTP server (using Gmail for this). Since GMail required SSL I needed to install and configure stunnel to communicate via SSL.