<object type="application/x-shockwave-flash" data="https://clients4.google.com/voice/embed/webCallButton" width="230" height="85"><param name="movie" value="https://clients4.google.com/voice/embed/webCallButton" /><param name="wmode" value="transparent" /><param name="FlashVars" value="id=e6e4e8cfd27c0ea62b30291a39b09fe06c5eeccc&style=0" /></object>
Monday, July 13, 2009
Call Me
Monday, July 6, 2009
Life Quotes | Life Quotations | Life Sayings | Wisdom Quotes
"If we could see the miracle of a single flower clearly, our whole life would change" -- Buddha
Friday, June 19, 2009
Thoughts on Oracle Data Integrator
I’ve been working fairly solidly with Oracle Data Integrator over the past few weeks, and I think I’ve used it enough now to form a few opinions on the product. I did some work with a client who wanted to evaluate it for it’s ability to handle Changed Data Capture, I’ve been writing an article for it for OTN and I’ve been scoping out the chapter on it for the forthcoming book. All of these tasks have required me to actually deliver something using it where I couldn’t just change the requirement if the tool didn’t do it well, so it’s been about as close an exercise as you can get to without actually using it on a real project. I’ve also had a chance to discuss how the tool is used with some of the ex-Sunopsis people who now work at Oracle, so hopefully I’ve got a good idea on how the tool is being positioned going in to the future.
Just to recap: Oracle Data Integrator is an ETL tool that Oracle acquired when it took over Sunopsis late last year. Data Integrator is being positioned as being the ETL tool to use as part of Oracle Fusion Middleware, as it’s just at home loading data into a non-Oracle database as an Oracle one, it can read from and publish to Web Services, it’s repository can reside on more or less any database, but it still uses the OWB paradigm of using the target database to do the ETL processing. In a way, it’s like OWB but written in a database-agnostic way, which I guess is why Oracle were interested in it in the first place. I wrote about ODI in it’s previous Sunopsis Data Conductor guise late last year, this link contains an archive of all the relevant blog postings.
If you drew a venn diagram of the functionality of OWB and the functionality of ODI, you’d get a considerable overlap, with the intersection being data integration, extracting from all different databases, loading to Oracle databases, data quality and error handling, support for OLAP concepts such as loading slowly changing dimensions and the creation of processes and workflows.
ODI would have as it’s unique capabilities the ability to load non-Oracle databases, native support for Changed Data Capture on both Oracle and non-Oracle databases, support for real-time ETL processes, built-in support for Web Services, and native support for extracting from non-Oracle databases (i.e. without the need for Oracle Gateways, or through ODBC). OWB would have, as it’s unique set of features, data modelling including OLAP data modelling, support for Oracle OLAP, a richer set of data mapping operators, the data profiler, transportable modules and change management/metadata change propagation.
So, one way of positioning ODI would be to say that it’s an alternative to OWB when performing the data integration and staging part of your warehouse build, particularly when you need to:
- Capture new and changed data from a database
- Source data from non-Oracle databases using native utilities
- Capture data from, or publish transformed data to, a Web Service
- Publish real-time ETL processes that respond to data pushed through them
One way of visualizing this is using the following diagram of the Oracle DW technology stack.
As you can see, ODI is in a way a potential replacement, or supplement, to the Enterprise ETL Option for Warehouse Builder - especially as, in the 10.2.0.1 and 10.2.0.2 release of OWB, Slowly Changing Dimension support has a number of bugs, and the Change Data Capture and Real-Time mappings features originally in the beta version were pulled prior to the production release. In this case, it makes a fairly good case for replacing at least some of the more complex and technical aspects of OWB’s functionality, at least in the stages prior to loading into the warehouse structures proper, and it’s even more of a compelling story if you’re dealing with non-Oracle sources and targets or indeed a SOA environment.
So, where ODI fits in with the existing Oracle DW stack is as a complement, or an alternative, to some of the functionality OWB provides up until you get to the ODS and analytic layers of the warehouse. Beyond that, that’s where OWB comes in to it’s own; the OLAP data modelling features, support for building Discoverer EULs and BI Beans presentations, support for loading AWs. These are areas ODI leaves well alone, which means that, one option going into the future is to use ODI together with OWB on your more complex data warehouse projects, perhaps paying for ODI ($12k per target database CPU, $4k per source database CPU) by foregoing the Enterprise ETL license ($10k per CPU on each database OWB is deployed to). I suspect Oracle would prefer you to license them both, but I’d say if your data integration needs are more complex than OWB can handle, consider ODI as a way of building the staging layer of the data warehouse, with OWB used to build the layers from the ODS up.
Data Quality is another area that both tools handle well, but differently. As you probably know, OWB now comes with the Data Quality Option, which apart from the well-known Data Profiler, also allows you to define error-handling on all your target objects with error rows neatly moved off into error handling tables. Although it uses PL/SQL and the Exceptions Into clause, in 10g this sort of code is just as fast - if not faster - than LOG ERRORS and it also allows you to keep all your error handling in one place, defined as data rules, rather than scattered all over your mappings.
ODI takes a slightly different approach. What it does is allows you to define “virtual” constraints on tables that ODI enforces, which are then monitored by Control Knowledge Modules with erroneous rows again moved off in to error tables. What’s good about ODI’s approach is that it effectively “firewalls off” erroneous data contained in source tables, such that they get copied on to error tables in a separate, workarea schema on the source database, and never get so far as to be loaded into the target environment.
This has the advantage of reducing the amount of data loading the target database has to carry out, as it doesn’t have to first load all the data, errors and all, into the target database before finding out which rows contain errors. This is particularly useful if your load window is small and you could do without unneccessarily processing rows that contain errors anyway - these are “firewalled off” by the virtual constraint and control knowledge module and never come near the target database. Where OWB comes in to it’s own though is the Data Profiler - with ODI, you need to know in advance what data rules (i.e. constraints) to apply, OWB on the other hand helps you determine them using the structure and semantics of your data.
Another area I found interesting with ODI was the support for Changed Data Capture. Some of the OWB old hands will know that CDC was originally going to be a feature of OWB10gR2, along with Real-Time mappings, but these were pulled from the product during the beta cycle, presumably so that they could actually get the product out of the door in a reasonable time. ODI however has pretty much full support for Changed Data Capture, both for Oracle databases and for other ones; for Oracle data sources, you can either capture changes using triggers, or using the Asynchronous Hotlog Changed Data Capture process I blogged about last year, with ODI acting as a graphical front-end for the process. I’m currently writing an article on this for OTN and so I won’t go into the full details, but in short the process for setting up CDC in this way on an Oracle 10g data source is as follows:
Firstly, enable your data server model (ODI terminology for an Oracle schema) for Journalizing, using either the simple trigger method, or the more featured consistent method that allows the capture of sets of tables using the LogMiner feature of Oracle 9i and 10g.
If you’re brave, ODI will even configure the source database to support Asynchronous Hotlog Change Data Capture, although it’s not recommended for production systems as it bounces the database half way through (that’ll keep your users/developers on their toes…)
Then, you select one or more tables to be part of the CDC set, and then use the Designer application to start the journal. Taking a look at the steps ODI takes to create the journal, you can see that it sets up CDC, in the standard way that Oracle 9i and 10g supports it, creates subscriber tables and sets up the capture and publish processes - pretty much all the things I did manually in my previous article.
Now I’ll be honest here, in that it took me a pretty long time to get this running properly, but the problems were more problems with the complexity of CDC on Oracle - getting the capture processes set up properly, clearing out failed setups when I was working out how to do it - rather than issues with ODI per se, although one annoying aspect of ODI did make things unneccesarily difficult. Basically, when you create your connections through to the source Oracle database, you naturally create them using the credentials of the source Oracle user account. The problem comes though when ODI tries to write data to the workarea schema it creates on the source database; it needs the CREATE ANY TABLE, SELECT ANY TABLE, DELETE ANY TABLE etc priviledges, together in fact with the DBA role if you want it to configure the database for you, which in production is a bit of a no-no. In fact, to get it working properly, I had to create the source database connection using the SYSTEM account credentials; the advice from Oracle is to do this initially, then set it back to the credentials of the source schema after setup is complete, but it was a bit of a hassle, and something that prolonged the exercise by about a week, to be honest. So the advice here is, especially if you’re using CDC - when you create your Oracle data server, use the SYSTEM account as the login (apparently the password is encrypted when stored in the repository, so it’s not the end of the world).
Anyway, once you’ve set up journaling, actually selecting from the new and changed data is a doddle - none of the fiddling around selecting from subscriber views that you get if you set it up yourself. You still need to extend and then purge the subscriber window, which you can do manually, or incoporate into a process flow, but then you just tick a box to say “work with journaled data” when adding the data store to an interface, and bob’s your uncle.
Beyond that point, you just deal with new and changed data the same way as data from the original table - a neat solution.
The obvious next question though, once you’ve got changed data capture working, is how to integrate the trickle of changes into your target database in real-time. In OWB, assuming you’d got CDC working through setting it up manually outside of OWB, you’d then probably design a mapping that read from the subscriber views and ran on a regular basis, say every minute or so, giving you latency of around a minute, depending on how long the CDC propagation and your mapping took. ODI takes a different approach though, in that you can define a process flow that reads from the journaled table, but then waits at the start for some data to appear in the journal before executing. Therefore, the process can be set to run, listen for changed data, and then kick-off when either a certain number of journal rows appear, or after a set period of time, whilst at the end the last step in the process kicks the process off again.
So, in conclusion, ODI looks like a pretty nifty additional toolset when working with OWB on a data integration project, or when building a data warehouse when the integration and staging element is non-trivial. If you need to deal with non-Oracle sources, or you need to capture changed data (possibly in real-time) on Oracle or any platform, or if you’re doing data integration in a SOA environment, it’s a handy box of tricks that makes the Oracle stack a more capable data integration platform. Of course when Oracle isn’t the target platform, it makes things possible that in reality just aren’t practical with OWB, and reading the tea leaves ODI is going to be the integration point for the metadata used for OBI EE and for loading data in to Hyperiod Essbase. In one sense, it could be seen as an alternative to Oracle Gateways or the Enterprise ETL Option for OWB, but in reality, I hope on projects in the future it’s an additional piece of kit that we use alongside OWB’s features, as I think the two tools complement each other, once you get to know what each of them does well.
Thanks
Sankar Chinnathambi
Herbalife International of America Inc.
Phone: 310-410-9600 Ext. 22359
Data Warehousing and changed data capture (CDC)
| Data Warehousing and changed data capture (CDC) |
| In an effort to capture the changes on some Oracle databases, we had to find out which is the best CDC tool with a low overhead load on the Production box and high throughput in terms of changes /sec. (Replication was used to avoid the load on the Production box, so that CDC can work on the Standby database). We have evaluated Oracle Streams and Oracle9i Data Guard and all these CDC tools works closely with the Oracle Logminer when you have to capture the changes from Oracle. Our experience with Oracle CDC and custom-writing the ETL code, lead us to the Oracle Data Integrator or ODI. The Oracle Data Integrator (ODI) is a data integration offering designed to help users make sense out of the incredible amount of information most frequently stored in various platforms and applications throughout an organization. ODI is optimized to work with Oracle's own database, application server, SOA suite and business activity monitoring software and the new version adds features that let users support packaged applications from IBM, Microsoft, Netezza, Sybase and others. The software also lets users build data warehouses that can be accessed and analyzed by Oracle Business Intelligence Suite Enterprise Edition (EE) 10g Release 3’s "hot-pluggable" or open-standards-based architecture gives it the ability to fit into any IT infrastructure and users can expect it to support the latest releases of IBM DB2, Microsoft SQL Server and NCR Teradata's data warehouse. Simply put, a "pervasive BI," that has the analytical capabilities that reach into all applications and data sources within an organization. One of the most important features of ODI is its ELT (extract, load, transform) Architecture -- not to be confused with ETL (extract, transform, load) technology, which Oracle calls "previous generation." With ELT, one can do transformations in the source or the target systems. One doesn't require a separate ETL server. You can do the transformation and push out the data to the target source. Another key feature of Data Integrator is its ability to support both batch and real-time data synchronization through its change-data-capture technology. What change-data-capture does is it allows us to monitor the log files of downstream legacy systems and databases to look for changing data. Once changing data is encountered these are raised as events to the Business Activity Monitoring tool natively, which in turn populates a dashboard." To summarize the ODI software helps users extract specific bits of information from the warehouse, build data marts, and conduct analyses to determine the effectiveness of sales and marketing and other initiatives. |
Thanks
Sankar
Tuesday, March 31, 2009
Tuesday, September 30, 2008
NRI Tax Info
A
A Rediff India Abroad feature:
My brother is working in
-- Jatin Shah
Subject to the local laws of the NRI, he or she is free to invest funds in equity, mutual funds, property and insurance related products in
The interest rates in
My daughter is a green card holder in the
Total earning from the above is less than the exempted limit of Rs 150,000. As per the bank's advice, she had submitted 15G form and tax was not deducted from April 1, 2007 to March 31, 2008 on the above investments. She has also not converted her accounts to Non-Resident Ordinary accounts.
1. Is it necessary to change the present bank accounts to NRO accounts?
2. Is it possible to change to NRO accounts from back date?
3. Is it necessary to deposit 30 percent of income earned through above investments during the year 2007-2008 now as income tax and then claim this amount from IT department by filing ITR.
-- Kanhaiya Lal
Yes, it is necessary to redesignate all her resident savings accounts to NRO. An NRI cannot have and operate resident accounts. It is not possible to do so retrospectively.
If her total income is less than Rs 150,000, then no tax is due from her and hence there is no requirement of filing a tax return.
I work on a ship. My employers have their head office in
-- Sharukh Cama
Tax is determined based on your residential status and not that of your employer's. Secondly, for a Resident Indian, global income is taxable. Therefore, since you have been in
I am an NRI for the last 12 years and have a Non-Resident External savings and NRO savings and fixed deposit accounts. I have a valid PAN Card too. I have no income from
Also, while going through the return form, I was not able to understand the requirement of Item 24 in the form. This item is described as 'Other Information (transactions reported through Annual Information Return) (Please see instruction number-9(ii) for code).' Could you elaborate on this also please?
-- Sundar
In
If any tax was payable originally, you would need to pay such tax and interest thereon @1% per month till the date that you finally pay the tax.
Regarding your other question, under section 285BA of the Income Tax Act, read with Rule 114E, the transactions of all persons, including NRIs, undertaking any one of the following seven specified financial transactions, equal to or over specified financial limits, are required to be reported to the IT department through Annual Information Return (AIR):
1. Cash deposits in a year in any bank SB account are of or more than Rs 10 lakh (Rs 1 million).
2. Payments in a year credit card are of or more than Rs 2 lakh (Rs 200,000).
3. Purchase of units of mutual funds of or more than Rs 2 lakh.
4. Acquisition of bonds or debentures of or more than Rs 5 lakh (Rs 500,000).
5. Purchase of shares of or more than Rs 1 lakh (Rs 100,000).
6. Purchase or sale of immovable property of or more than Rs 30 lakh (Rs 3 million).
7. Purchase of RBI bonds of or more than Rs 5 lakh.
If you do not have any of such transactions, you may write 'nil' in the space provided.
I am a marine engineer (Merchant Navy) having permanent residence in Kolkata. What will be my tax status and tax treatment? Will I be a NRI even though I sail for less than six months an year? Can I hold a normal savings account along with NRE account in
-- Saurav Agarwal
If any person spends 182 days or more in
One can either be a Resident or an NRI. Accordingly, one can either have a Resident savings account or an NRE / NRO account. As a resident individual, you can directly invest in the stock market. An NRI to do the same would require the PINS permission.
Thanks
Sankar Chinnathambi
Senior Manager - Oracle Applications
Herbalife International of America Inc.
Phone: 310-410-9600 Ext. 22359
Wednesday, September 17, 2008
Parents: When to back off, when to step in
Parents: When to back off, when to step in
Do you speed over to your child's school, or even college, whenever something goes wrong?
Parents should try to teach their children how to handle situations themselves, experts say.
Many parents today insert themselves into even the most minute activities in their children's lives, a phenomenon that's known as "helicopter parenting."
But two child experts told CNN that parents should aim to empower their children to do things on their own.
"Parenting should be increasingly in the background as the child gets older," said Vivian Friedman, child-adolescent psychologist at the
Dr. Kenneth Ginsburg, M.D., associate professor of pediatrics at the University of Pennsylvania School of Medicine and author of the book "A Parent's Guide to Building Resilience in Children and Teens," said it's important to show children you believe they are capable of handling situations themselves, but always put safety first. 
The "helicopter parent" trend may have arisen because the focus of marriage has shifted from the spouse to the child, Friedman said. An increase in divorce and a greater prevalence of two working spouses also contribute to parents' habits of spoiling their children.
"When [the kids] have little failures, they feel like our own failures," Ginsburg said. "What we need to understand is that our job as parents is not to finish our kids or produce perfect kids. It's to start our kids." 
Here's what Friedman and Ginsburg recommended for various situations that may arise in your child's life:
1. The elementary school is putting on "Peter Pan" tomorrow. While you prepare a presentation for an important business meeting, your daughter calls and tells you she forgot her Tinkerbell costume for the dress rehearsal. "I'm the only one who's not dressed up," she tells you.
Friedman and Ginsburg agreed that it's OK to take care of it the first time she forgets the costume, but not if it becomes a habit.
"The first time: Find a housekeeper or neighbor to bring it over," Friedman said. "But if she does this routinely, she needs to suffer consequences to learn from the experience."
Don't Miss
How to ground a helicopter parent
"One time, two-time mistake -- it's wonderful to pick up the pieces," Ginsburg said. "If, in fact, you always end up picking up pieces, you can't expect a kid to learn the valuable lesson that they can do it themselves. Learn from failure -- failure's a great thing when you learn how to recover yourself."
2. You're at the playground reading a newspaper and suddenly your daughter runs up to you crying. "They won't let me make sand castles with them," she whines.
Friedman said this one depends on age. For a 3- or 4-year-old, it's appropriate for the parent to go over to the group as a neutral adult and help the children learn that they need to include everybody.
But for a 7-year-old, it's borderline. "You could say, 'What seems to be the problem? Do you think there's a way we could all play together?' rather than 'You can't exclude my child,' " she said.
Ginsburg, on the other hand, said you can suggest to your child what to say, or recommend that she find someone else to play with, but you shouldn't communicate that your child isn't capable of handling the situation.
"Learning how to play nice with other people in the sandbox is a great metaphor for life," he said.
3. Kids in the seventh-grade class just won't leave your son alone, not even online. Besides shoving him against the lockers once in awhile, they've also set up a MySpace page making fun of his appearance and name.
Experts agree that you should step in here and notify the school. Friedman would also notify the parents of the children involved -- "Most rational, reasonable parents would not support their child doing that," she said.
Ginsburg emphasized that safety always comes first. "You don't allow your daughter to put her hand in the oven to learn it's hot," he said. "You don't allow a kid to be bullied when there should be systems in place to prevent bullying. "
4. Your son started college a month ago, and every time you call him he has a new story about his messy, party-loving roommate who distracts him from studying and interrupts his sleep. "I wouldn't mind as much if it were my own vomit on the floor," your son tells you. He says he'll just stick it out for the rest of the year.
Health Library
MayoClinic.com: Children's Health
Friedman and Ginsburg said they would not approach the college housing department and ask to have the young man moved, except in extreme or difficult circumstances -- such as if the child is in a special needs program and can't help himself, Friedman said.
"There's nothing wrong with asking open-ended questions to help your son figure it out," Ginsburg said. "It's OK to say 'Who can you talk to at the university to change your living situation?' What's not OK is to call the dean and say 'Move my son.' " 5. Always a technical genius, your daughter majors in electrical engineering and will surely become a pioneer of great innovations. But this semester, the last of her junior year, she failed her 18th-century literature class, which she took to fulfill the subject-area distribution requirements. This is going to look pretty bad on graduate school applications.
While the daughter can speak to the professor on her own, Friedman and Ginsburg said they would not intervene in this situation.
Special circumstances would be if the grade was truly unfair and there was real foul play involved, or if the professor was drunk -- but otherwise, Friedman said, "It's her F, it's not your F. I would do absolutely nothing."
6. Your daughter has been in the real world for a year, but she says she's not ready to keep herself afloat financially. For the third time this summer she asks for help paying the rent -- "phone bill would be extra nice," she adds. You also notice that she's got a new pair of Manolo Blahnik pumps, and an iPhone is sticking out of her purse.
Friedman said she would not allow this to come up a third time -- she would have made the daughter set up an automatic debit system early on so her child's rent comes out of the account when her paycheck is deposited. "By the third month, I would let her sink, but I'm not a helicopter parent," she said.
Ginsburg emphasized again the safety component: He would never want his daughter to become homeless. He would pay the first month's rent with clear expectations: She needs to learn how to make a budget, she can't spend money on other things until things like rent and food are taken care of, and she needs to know that this is a loan. "Seven months in a row: she needs to find a roommate," he said.
Finally, note that there are no villains here, Ginsburg said.
"The parents who we think do too much are still doing their very best," he said. "Real success involves resilience: the capacity to learn to bounce back on your own."
Subscribe to:
Posts (Atom)