Showing posts with label Tuning. Show all posts
Showing posts with label Tuning. Show all posts

Wednesday, December 02, 2015

Displaying Oracle 10046 Trace File in SQL developer


Good know that SQL Developer can display 10046 Trace file in nice view,  as an alternative to using the TKPROF program.
To open a .trc file in SQL Developer, click File/Open, and choose the desired trace file.

Once opened, we can examine the information in the Tree View,  Statistics View, and List View.

Below is my sample screenshots.

BTW, it seems List View requires large Graphic card memory.





Thursday, September 01, 2011

Partition Pruning Links

Partition Pruning

Partition Pruning and Joins [ID 179518.1]

Ask Tom "partition pruning"  

Friday, August 05, 2011

Hash Group by exceed temporary tablespace

Tuned a "HASH GROUP BY" after increase the pga_aggregate_target from 190MB to 2GB , since workload increased over the time.

Feeling the data volume in below plan :

- massive SUMs
- 52 milliows row,
- 5 month's data.
PERFSTAT@ABC>@disp_cursor_plan                                                                                             
Enter value for sqlid: 885gs8h6jjxzc                                                                                         
old   1: select * from table(dbms_xplan.display_cursor('&sqlid'))
new   1: select * from table(dbms_xplan.display_cursor('885gs8h6jjxzc'))

SQL_ID  885gs8h6jjxzc, child number 0
-------------------------------------
       ,sum(CASE period WHENesets
to_date('201107','YYYYMM') THEN (case hourofday when 0 then mou else 0 end) ELSE 0
       ,sum(case hourofday when 0 then mou else 0 end) as
       ,sum(CASE period WHEN to_date('201107','YYYYMM') THEN (case
hourofday when 0 then no_call else 0 end) ELSE 0 END) AS nocall1m_0001
,sum(case hourofday when 0 then no_call else 0 end) as nocall6m_0001
,sum(CASE period WHEN to_date('201107','YYYYMM') THEN (case hourofday when 1 then
       ,sum(case hourofday when 1 then102
       ,sum(CASE period WHEN2
to_date('201107','YYYYMM') THEN (case hourofday when 1 then no_call else 0 end)
       ,sum(case hourofday when 1 then no_call else 0
       ,sum(CASE period WHEN to_date('201107','YYYYMM') THEN
(case hourofday when 2 then mou else 0 end) ELSE 0 END) AS mou1m_0203

Plan hash value: 2842556147

--------------------------------------------------------------------------------------------
| Id  | Operation          | Name                  | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |                       |       |       |   263K(100)|          |
|   1 |  HASH GROUP BY     |                       |    52M|  3929M|   263K  (8)| 00:52:48 |
|*  2 |   TABLE ACCESS FULL| DM_TRAN_LOCALOUTGOING |    52M|  3929M|   253K  (4)| 00:50:43 |
--------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter(("CALL_TYP"='VOICE' AND "HOUROFDAY">=0 AND "HOUROFDAY"<=23 AND
              "PERIOD"<=TO_DATE(' 2011-07-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss') AND
              "PERIOD">=TO_DATE(' 2011-02-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss')))

Note
-----
   - dynamic sampling used for this statement


37 rows selected.


Hash group (and hash joins, as well as other operations such as sorts etc.) can use either optimal (i.e. in-memory), one-pass or multi-pass methods. The last two methods use TEMP storage and is thus much slower.
By increasing the number of possible items you might have exceeded the number of items that will fit in memory reserved for this type of operations.
Try looking at v$sql_workarea_active whilst the query is running, to see if this is the case. Or look at v$sql_workarea for historical information. It will also give you an indication of how much memory and/or temp space is needed for the operation.
If turns out to be the actual problem - try increasing the pga_aggregate_target initialization parameter, if possible. The amount of memory available for optimal hash/sort operations is usually around a 5% fraction of the pga_aggregate_target.


Hash Join

   The hash join is used for high-volume equi-joins (joins with equals predicates). Oracle performs a single read of the smaller row source (call this T1) and builds a hash table in memory. The join key is used as the hash-key of the hash table. Then a single pass of the larger row source (call this T2) is performed, hashing the join key of each row to obtain an address in the hash table where it will find matching T1 rows.
  Provided T1 remains small enough to build the hash table in memory, T2 can be scaled up to any arbitrarily large volume without affecting throughput or exceeding temp space. If T1 cannot be hashed in memory, then a portion of the hash-table spills to disk. When the hash table is probed by T2, the rows with join keys that match those parts of the in-memory hash table are joined immediately; the rest are written to TEMP and joined in a second pass. The bigger T1 is, the smaller the proportion of the hash table that can fit in memory, and the larger the proportion of T2 that must be scanned twice. This slows the Hash Join down considerably and also makes the join non-scalable.

For large hash processes, where temp tablespace usage is unavoidable, temp tablespace groups offer significant improvement in I/O performance and more CPU utilization.

 Indexed Nested Loops is used primarily in low volume joins; it is efficient over small volumes and versatile enough to be used in a variety of situations. Although it is fully scalable, Indexed Nested Loops is inefficient over large data volumes.

2.8 millions  rows in driving table looks up a larger table still should be regarded as inefficient. I had once case which is unable to complete running for 12 hours. Solved by using hash join plus partition pruning. 

Sort-Merge

A sort-merge join works by reading each row-source in the join separately; sorting both sets of results on the join column(s); then concurrently working through the two lists, joining the rows with matching keys. Sort-Merge is generally faster than Indexed Nested Loops but slower than Hash Join for equi-joins. It is used almost exclusively for non-equi joins (>, <, BETWEEN) and will occasionally be used when one of the row sources is pre-sorted (eg. a GROUP BY inline view)
If both row sources are small then they may both be sorted in memory, however large sorts will spill to disk making then non-scalable.
There is no way to make a Sort-Merge join scalable. The only other way to resolve a non-equijoin is to use Nested Loops, which is slower. As volumes increase, Sort-Merge will continue to out-perform Nested Loops, but will eventually run out of Temp space. The only solution is to extend TEMP, or convert the join to Nested Loops (and then wait).


 also   read
http://www.dbspecialists.com/files/presentations/temp_space.html
http://stackoverflow.com/questions/154722/can-anyone-explain-how-the-oracle-hash-group-works 

Friday, November 12, 2010

Using advantage of partition elimination

[The problem]

    Application team asked for help:
  • Daily financial report delayed near one month because job running very slow while DB to shutdown everyday for cold backup. 
  • My estimation is about 30 hours for the job to completed. 
  • Vendor not able to provide solution even tried changing code a few times.



[Diagnostic]
    This range partitioning table is about 140Gb big, with 500+ partitions, even partition stores 5 values of job_id.
    From execution plan, partition is performed but no parallelism regardless PARALLEL server enabled, and no performing full table scan, how ever generating lots of I/O , consist gets requires , physical reads etc.
    Total consist gets is about 5 times of partitions needed for scan.
    The statement is like this:
     select ... from p_table where part_key_col in (select distinct job_id from job_table...);

    The execution plan shows NEST LOOP for each value return from sub-query, caused redundantly access to partitions.
    ie.
     for each job_id returned from sub-query (110 distinct job_id)
       do
         full partition scan  ( one of total 22 partitions )
       done.
  Hence, each partition is scanned 5 times in worst situation. Total times: 110
  While parallelized scan can't happen in single partition, which only occurs for simultaneoustly access to multiple partitions.

   
[Solution]
    Rewrite the code , to make partition elimination happen.
    select  ... from p_table where part_key_col between (select min(distinct job_id) from job_table ...) and (select max(distinct job_id) from job_table ...);
    Not that the logic slightly changed, but applicable in this case.

     After make this change according to my suggestion, job finished within 30 minutes, while observing 12 parallel processes running happily to scan 22 partitions once only.
    Cheers!

[Update on 17-Nov]

One more think,  SQL logic should not be changed.  Studied more about partition pruning from data warehousing guide, found USE_HASH hint achieved same effects without rewrite SQL.

Sunday, April 18, 2010

two cases of missing indexes

Case 1:

SQL> select status, count(*) from dbaantispam.tb_sms_out group by status;

    STATUS   COUNT(*)
---------- ----------
         1     112459


SQL> alter session set current_schema=dbaantispam;

Session altered.

SQL> set autotrace on
SP2-0613: Unable to verify PLAN_TABLE format or existence
SP2-0611: Error enabling EXPLAIN report

cd  $ORACLE_HOME/rdbms/admin/

SQL> @utlxplan

Table created.

SQL> set autotrace on

SQL> conn /
Connected.
SQL> set autotrace on


SQL>  SELECT sms_out_id, text, msisdn, messagetype FROM dbaantispam.tb_sms_out WHERE status = 0 ORDER BY sms_out_id ASC;

no rows selected


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'TB_SMS_OUT'
   2    1     INDEX (FULL SCAN) OF 'SMS_OUT_ID_PK' (UNIQUE)




Statistics
----------------------------------------------------------
        248  recursive calls
          0  db block gets
      31466  consistent gets
       3072  physical reads
          0  redo size
        467  bytes sent via SQL*Net to client
        461  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          6  sorts (memory)
          0  sorts (disk)
          0  rows processed

As you can see there are lot of  consistent reads and physical reads .

SQL> @chk_indcol
Enter value for tbl: TB_SMS_OUT
old   1: select index_name,column_name,column_position from dba_ind_columns where table_name=upper('&tbl')
new   1: select index_name,column_name,column_position from dba_ind_columns where table_name=upper('TB_SMS_OUT')

INDEX_NAME                     COLUMN_NAME          COLUMN_POSITION
------------------------------ -------------------- ---------------
SMS_OUT_ID_PK                  SMS_OUT_ID                         1

-- No index found on STATUS column
SQL> create index dbaantispam.tb_sms_out_statusidx on tb_sms_out(status) tablespace antispam_idx;

Index created.



SQL>  SELECT sms_out_id, text, msisdn, messagetype FROM dbaantispam.tb_sms_out WHERE status = 0 ORDER BY sms_out_id ASC;

no rows selected


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   SORT (ORDER BY)
   2    1     TABLE ACCESS (BY INDEX ROWID) OF 'TB_SMS_OUT'
   3    2       INDEX (RANGE SCAN) OF 'TB_SMS_OUT_STATUSIDX' (NON-UNIQ
          UE)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          2  consistent gets
          1  physical reads
          0  redo size
        467  bytes sent via SQL*Net to client
        461  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
          0  rows processed

2 consistent reads and 1 physical reads only.



Case 2 :


alter session set current_schema=dbamsgctr;


SQL> set timing on
SQL> set pages 1000
SQL> SELECT '11:00-11:59', COUNT(destination_type) FROM dbamsgctr.tb_msg_transaction
  2   WHERE destination_type = 0 AND msg_type LIKE '%SMS%'
  3   AND transaction_timestamp >= TO_DATE('2010-03-16 11:00:00', 'yyyy-mm-dd hh24:mi:ss')
  4   AND transaction_timestamp <= TO_DATE('2010-03-16 11:59:59', 'yyyy-mm-dd hh24:mi:ss');


'11:00-11:5 COUNT(DESTINATION_TYPE)
----------- -----------------------
11:00-11:59                       0

Elapsed: 00:07:21.94

Execution Plan
----------------------------------------------------------




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     380503  consistent gets
     380195  physical reads

          0  redo size
        586  bytes sent via SQL*Net to client
        656  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

Huge amounts of logical reads, because of this statement, during 3 hours ,  the "Buffer Cache Reads" is 278.4G

SQL> set timing on pages 1000
SQL> create index msg_idx2  on tb_msg_transaction(transaction_timestamp) tablespace msgctr_idx nologging parallel (degree 3 );


Index created.

Elapsed: 00:14:55.86

alter index msg_idx2 logging noparallel;



SQL> set autotrace on
SQL> SELECT '11:00-11:59', COUNT(destination_type) FROM dbamsgctr.tb_msg_transaction
  2   WHERE destination_type = 0 AND msg_type LIKE '%SMS%'
  3   AND transaction_timestamp >= TO_DATE('2010-03-16 11:00:00', 'yyyy-mm-dd hh24:mi:ss')
  4   AND transaction_timestamp <= TO_DATE('2010-03-16 11:59:59', 'yyyy-mm-dd hh24:mi:ss');

'11:00-11:5 COUNT(DESTINATION_TYPE)
----------- -----------------------
11:00-11:59                       0

Elapsed: 00:00:02.81

Execution Plan
----------------------------------------------------------




Statistics
----------------------------------------------------------
        164  recursive calls
          0  db block gets
         28  consistent gets
          4  physical reads

          0  redo size
        586  bytes sent via SQL*Net to client
        656  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          6  sorts (memory)
          0  sorts (disk)
          1  rows processed

Tremendous improvement !      minutes of run becomes 3 seconds !!! 
Again , housekeeping and Oracle developer understands  how Oracle database works is important.

Saturday, March 20, 2010

Execute to Parse

I’m confused about the execute-to-parse ratio in Statspack/automatic workload repository reports. I have observed this ratio as 12.02 percent. My team is suggesting a change to the CURSOR_SHARING parameter value—to SIMILAR—to improve this ratio. Is that a correct approach? Can you please explain what this ratio actually means?

The last part of that question should throw up a bunch of red warning flags. You are looking at a ratio: the execute-to-parse ratio. You feel it is “bad” and would like to fix it, so you suggest changing a parameter to fix it. But then you ask for the ratio to be explained, meaning you’re not really sure what it is, what it means, and if the value you have is bad. Not only that, but the suggestion to change CURSOR_SHARING cannot and will not change the execute-to-parse ratio.

OK, first of all, let’s start with an explanation of what the execute-to-parse ratio is. It is a measure of how many times, on average, your SQL statements are executed rather than parsed. The way this ratio is computed, it will be a number near 100 percent when the application executes a given SQL statement many times over but has parsed it only once. (An application must parse a SQL statement at least once to execute it but needs to parse the SQL statement only once to execute it over and over again.) This ratio will be near 0 percent if the application parses a statement every time it executes it (parse count = execute count). This ratio will go negative if the application parses more often than it executes (and that would definitely be a sign that something is seriously wrong in the application). The formula (from Statspack) is simply

'Execute to Parse %:’, round(100*
(1-:prse/:exe),2)

As you can see, if :prse (parse count) is about the same as :exe (execute count), the execute-to-parse ratio will be 0 percent. If :exe is much larger than :prse, :prse/:exe will be near zero and the execute-to-parse ratio itself will be near 100 percent. If :prse is greater than :exe, the execute-to-parse ratio will go negative (indicating “this is bad”).

So that is, technically speaking, what the ratio is. A number near 100 percent would be great but might not be attainable. A negative number should definitely be avoided—it would indicate that the application actually parses a SQL statement but never executes it. The application developers would need to be educated, because a parse is an expensive operation and their goal is to reduce the number of parse calls—not create extra, unnecessary ones!

So what about the observed percentage, 12 percent? There is room for improvement there, but it cannot come from CURSOR_SHARING. Changing that parameter from its default (and preferred) setting of EXACT to SIMILAR might change the type of parsing happening (the parameter change can convert a hard parse into a soft parse), but it will never reduce the number of parse calls made by the application. After the parameter change, the application will still be calling “prepare statement” to parse SQL—under the covers, it might be a soft parse, but it will still be a parse.

In short, your colleagues’ suggestion would do nothing to alter the execute-to-parse ratio and could, in fact, cause major issues in your already-running system. Don’t even consider changing the CURSOR_SHARING parameter in order to try to fix your execute-to-parse ratio.

The only way to modify the execute-to-parse ratio would be to alter the two variables used in the formula. You can change either (1) the number of times you parse or (2) the number of times you execute.

I vote for the first option. Your application should look for ways to reduce the number of times it parses SQL, and how to do this varies from environment to environment. Using JDBC, for example, you can enable JDBC statement caching (a quick search for “JDBC statement cache” will turn up plenty of information). Using .NET, you can also enable statement caching (a quick search for “Oracle .NET statement cache” will turn up lots of information as well).

My preferred way to improve your execute-to-parse ratio, however, is to move all the SQL out of the client application and into stored procedures. PL/SQL is a statement caching machine—it has, from its very beginning, kept a cache of cursors open for us. When you say “close this cursor” in PL/SQL, PL/SQL tells you, “OK, it’s closed,” but it really isn’t. The PL/SQL engine smartly keeps the cursor open, knowing that you are just going to call that stored procedure and execute that SQL again.

Suppose you have a subroutine in your client application that is called 10,000 times a day. Further, suppose that it executes five SQL statements and those statements are parsed and executed every single time that routine runs. You will be doing 50,000 parse calls and 50,000 executes.

If you were to move those five SQL statements into a stored procedure and execute just one PL/SQL call in the client application—even assuming that you didn’t cache that statement in the client—you would now have

  • 10,000 parse calls for the PL/SQL block
  • 5 parse calls for the 5 SQL statements
  • 10,000 execute calls for the PL/SQL block
  • 50,000 execute calls for the SQL in the PL/SQL code

So now you will have 10,005 parse calls and 60,000 executes (plus many fewer round-trips between client and server). The parse calls (be they hard or soft parses) are extremely expensive. Having this one application instance cut down the number of parse calls to 20 percent of what it used to be will have an impact on performance and scalability—a profound impact.

In short, the only way to really affect the execute-to-parse ratio is to change the number of execute or parse calls—and this is something the application developer has to do. No magic parameter can be set.

Tuesday, November 10, 2009

Temporary segments do not span tablespaces of Temporary Tablespaces Group

I took for granted that single session with one big sort can benefit from tablespace group.
However, it is wrong ! No wonder our data warehouse often hit ora-1652.
--before make change , temp1,2,3 size is 16gb,16gb,19gb respectively
--This is means available temporary range is 16gb to 19gb.

The relevant metalink doc is 245645.1 and 248712.1

Reason is quite simple , Temporary segments do not span tablespaces. This easy to understand, same as other segment.


-- change support id to use small temp3 , which is the default database temporary tablespace (check from database_properties table)

-- remove temp1, temp2,temp3 from temp_group

alter tablespace TEMP1 tablespace group '';
alter tablespace TEMP2 tablespace group '';
alter tablespace TEMP3 tablespace group '';

--drop temp2

drop tablespace temp2 including contents and datafiles;

--epxand temp1;

--shrink temp3

--after make change , temp1,3 size is 40gb,10gb respectively
--assign application id to use big temporary tablespace


Since we don't parallel DML & have limited diskspace, tablespace group does't help.

From this practice, I think tablespace group is only good if you have lots of tablespace to create tablespaces with same size for round-robin assignment fashion.
Ideally, can create them on separate disks to reduce I/O contention.


How come important things is missed out in many articles searched by Google !?

Sunday, June 08, 2008

Friday, May 30, 2008

Merge Join caused Cartesian Product

--Merge Join caused Cartesian Product

1591 RPTOPR CREATE TABLE tmp_aio_acct_tp1 as SELECT distinct(a.customer_id),
1591 RPTOPR decode((c.bld_stat_id),1,'Blacklisted') blist_status, d.bdr_evs
1591 RPTOPR dt start_blist, d.bdr_blist_resn_type_id blist_reason FROM tmp_a
1591 RPTOPR io_acct_tb a, blist_cust b, blist_dtls c, blist_dtls_resn_code d
1591 RPTOPR -- WHERE rtrim(a.customer_id) = rtrim(b.blc_cust_id) and WHERE
1591 RPTOPR rtrim(a.customer_id) = b.blc_cust_id and b.bld_blist_id=c.bld_bl
1591 RPTOPR ist_id and b.bld_blist_id=d.bld_blist_id and d.bdr_evedt is NULL
1591 RPTOPR


--this query can't finish event after 10 hours.
--expect 16 hours to completed.
--notice the executions is exterm high.

^LSQL ordered by Executions DB/Inst: PRXP/PRXRP Snaps: 581-587
-> Total Executions: 96,444,779
-> Captured SQL account for 100.0% of Total

CPU per Elap per
Executions Rows Processed Rows per Exec Exec (s) Exec (s) SQL Id
------------ --------------- -------------- ---------- ----------- -------------
96,183,306 0 0.0 0.00 0.00 b9ag273wzuhbx
Module: oracleODSP@ods01 (TNS V1-V3)
SELECT "BLC_CUST_ID","BLD_BLIST_ID" FROM "DBO"."BLIST_CUST" "B" WHERE "BLC_CUST_
ID"=:1 AND "BLD_BLIST_ID"=:2


Elapsed CPU Elap per % Total
Time (s) Time (s) Executions Exec (s) DB Time SQL Id
---------- ---------- ------------ ---------- ------- -------------
2,900 2,898 96,183,306 0.0 14.4 b9ag273wzuhbx
Module: oracleODSP@ods01 (TNS V1-V3)
SELECT "BLC_CUST_ID","BLD_BLIST_ID" FROM "DBO"."BLIST_CUST" "B" WHERE "BLC_CUST_
ID"=:1 AND "BLD_BLIST_ID"=:2


^LSQL ordered by Gets DB/Inst: PRXP/PRXRP Snaps: 581-587
-> Resources reported for PL/SQL code includes the resources used by all SQL
statements called by the code.
-> Total Buffer Gets: 719,230,530
-> Captured SQL account for 99.9% of Total

Gets CPU Elapsed
Buffer Gets Executions per Exec %Total Time (s) Time (s) SQL Id
-------------- ------------ ------------ ------ -------- --------- -------------
319,369,588 96,183,306 3.3 44.4 2897.99 2900.45 b9ag273wzuhbx
Module: oracleODSP@ods01 (TNS V1-V3)
SELECT "BLC_CUST_ID","BLD_BLIST_ID" FROM "DBO"."BLIST_CUST" "B" WHERE "BLC_CUST_ID"=:1 AND "BLD_BLIST_ID"=:2



1591 RPTOPR 1 0 1 LOAD AS SELECT
1591 RPTOPR 2 1 1 SORT
1591 RPTOPR 3 2 1 NESTED LOOPS
1591 RPTOPR 4 3 1 NESTED LOOPS
1591 RPTOPR 5 4 1 MERGE JOIN
1591 RPTOPR 6 5 1 REMOTE BLIST_DTLS_RESN_CODE
1591 RPTOPR 7 5 2 BUFFER
1591 RPTOPR 8 7 1 TABLE ACCESS TMP_AIO_ACCT_TB
1591 RPTOPR 9 4 2 REMOTE BLIST_CUST
1591 RPTOPR 10 3 2 REMOTE BLIST_DTLS


-- the rows of these two tables explains the high number of executions.
-- 3 millions rows in BLIST_DTLS_RESN_CODE and one quarter records meet "d.bdr_evedt is NULL", 7k rows in TMP_AIO_ACCT_TB
-- 3200k/4 * 7k = 5600k * k = 5600 Millions



SELECT distinct(a.customer_id),
decode((c.bld_stat_id),1,'Blacklisted') blist_status, d.bdr_evsdt start_blist, d.bdr_blist_resn_type_id blist_reason
FROM t1 a, blist_cust b, blist_dtls c, blist_dtls_resn_code d
WHERE rtrim(a.customer_id) = b.blc_cust_id and b.bld_blist_id=c.bld_blist_id and b.bld_blist_id=d.bld_blist_id and d.bdr_evedt is NULL;

SELECT /*+ ordered use_nl(b c d) */
distinct(a.customer_id),
decode((c.bld_stat_id),1,'Blacklisted') blist_status, d.bdr_evsdt start_blist, d.bdr_blist_resn_type_id blist_reason
FROM t1 a, blist_cust b, blist_dtls c, blist_dtls_resn_code d
WHERE rtrim(a.customer_id) = b.blc_cust_id and b.bld_blist_id=c.bld_blist_id
and b.bld_blist_id=d.bld_blist_id and d.bdr_evedt is NULL;

1601 RPTOPR 0 17168 SELECT STATEMENT
1601 RPTOPR 1 0 1 SORT
1601 RPTOPR 2 1 1 NESTED LOOPS
1601 RPTOPR 3 2 1 NESTED LOOPS
1601 RPTOPR 4 3 1 NESTED LOOPS
1601 RPTOPR 5 4 1 TABLE ACCESS T1
1601 RPTOPR 6 4 2 REMOTE BLIST_CUST
1601 RPTOPR 7 3 2 REMOTE BLIST_DTLS
1601 RPTOPR 8 2 2 REMOTE BLIST_DTLS_RESN_CODE



-- finished within 1 mins



--try this hint (bad plan same as orginal)
SELECT /*+ use_nl(b c d) */
distinct(a.customer_id),
decode((c.bld_stat_id),1,'Blacklisted') blist_status, d.bdr_evsdt start_blist, d.bdr_blist_resn_type_id blist_reason
FROM t1 a, blist_cust b, blist_dtls c, blist_dtls_resn_code d
WHERE rtrim(a.customer_id) = b.blc_cust_id and b.bld_blist_id=c.bld_blist_id
and b.bld_blist_id=d.bld_blist_id and d.bdr_evedt is NULL;


1601 RPTOPR 0 4506 SELECT STATEMENT
1601 RPTOPR 1 0 1 SORT
1601 RPTOPR 2 1 1 NESTED LOOPS
1601 RPTOPR 3 2 1 NESTED LOOPS
1601 RPTOPR 4 3 1 MERGE JOIN
1601 RPTOPR 5 4 1 REMOTE BLIST_DTLS_RESN_CODE
1601 RPTOPR 6 4 2 BUFFER
1601 RPTOPR 7 6 1 TABLE ACCESS T1
1601 RPTOPR 8 3 2 REMOTE BLIST_CUST
1601 RPTOPR 9 2 2 REMOTE BLIST_DTLS

-
--tried ordered only

SELECT /*+ ordered */
distinct(a.customer_id),
decode((c.bld_stat_id),1,'Blacklisted') blist_status, d.bdr_evsdt start_blist, d.bdr_blist_resn_type_id blist_reason
FROM t1 a, blist_cust b, blist_dtls c, blist_dtls_resn_code d
WHERE rtrim(a.customer_id) = b.blc_cust_id and b.bld_blist_id=c.bld_blist_id
and b.bld_blist_id=d.bld_blist_id and d.bdr_evedt is NULL;

1601 RPTOPR 0 9458 SELECT STATEMENT
1601 RPTOPR 1 0 1 SORT
1601 RPTOPR 2 1 1 HASH JOIN
1601 RPTOPR 3 2 1 REMOTE BLIST_DTLS_RESN_CODE
1601 RPTOPR 4 2 2 HASH JOIN
1601 RPTOPR 5 4 1 TABLE ACCESS T1
1601 RPTOPR 6 4 2 REMOTE


--okay



--think driving_site should also work, but have no chance to test again.


The join operations group of hints controls how joined tables merge data together. A join
operation may direct the optimizer to choose the best path for retrieving all rows for a query
(throughput) or for retrieving the first row (response time).
while ORDERED tells the optimizer to join the tables based on their
order in the FROM clause using the first table listed as the driving table (accessed first).

--tuned queries from hours to seconds using this method


However, as to root casue , optimizer is still a black box to us!

Thursday, May 22, 2008

redo log size

CHECK V$MTTR_TARGET_ADVICE and v$log_history for log switch ,
and optimal log size , which is influced by mttr target

The view V$INSTANCE_RECOVERY contains a new column, OPTIMAL_LOGFILE_SIZE,
which recommends a minimum size for the redo log files:
SQL> select optimal_logfile_size from v$instance_recovery;

Monday, May 12, 2008

example of index suppressed

The conversion function to_number(), causes existing index not chosen.

use to_char function on the right side instead. i.e

WHERE CDT_ACCT_NBR = to_cahr(:1)


SQL> @chk_sqltext

933 DBLN_RPTP SELECT "CDT_ACCT_NBR","CDT_COMPANY_NAME" FROM "DBO"."CUST_DTLS"
933 DBLN_RPTP "B" WHERE TO_NUMBER("CDT_ACCT_NBR")=:1

SQL> @chk_sqlplan

933 DBLN_RPTP 0 3699 SELECT STATEMENT
933 DBLN_RPTP 0 3699 SELECT STATEMENT
933 DBLN_RPTP 1 0 1 TABLE ACCESS CUST_DTLS
933 DBLN_RPTP 1 0 1 TABLE ACCESS CUST_DTLS

SQL> @chk_indcol
Enter value for tbl: CUST_DTLS
old 1: select index_name,column_name,column_position from dba_ind_columns where table_name='&tbl'
new 1: select index_name,column_name,column_position from dba_ind_columns where table_name='CUST_DTLS'

PK_CUST_DTLS
CDT_ACCT_NBR
1


Saturday, May 10, 2008

Effect of creating missing index

From AWR report of one OLTP database, I identified three missing index based on top buffer gets and physical reads. After creating them with eager, the outcome is tremendous to me -almost improved 100%.

Comparing Period: 3pm ~ 6pm of 8-May and 9 -May (in seconds)

DB Time (=response time?) : 39317s -- > 20526s (database become less busy)
Service Time: 23709s --> 10767s ( less CPU time)
Wait Time: 15608s -- > 9759s (DB Time - Service Time)
Buffer Hit Ratio: 75% --> 89% (some more to tune)



TIP
Tuning the top 25 buffer get and top 25 physical get queries has
yielded system performance gains of anywhere from 5 percent to
5000+ percent in my tuning. The SQL section of the STATSPACK
report tells you which queries to consider tuning first. The top 10 SQL
statements should not be substantially more than 10 percent of your
buffer gets or disk reads.

Wednesday, April 23, 2008

如何学习ORACLE优化

反正有空,说一下我能想到的,很多方面我的了解都很浅薄,甚至只有一个不甚清楚的概念,仅供参考。

最基本的还是要了解oracle的工作方式,oracle的架构,存储的,内存的,进程的,dedicate server/shared server, redo和undo,latch和lock。。。。。。no knowledge, no tune.

了解常见的调优思想,responese time=wait time+service time是最基本的思想,基于ratio的思想虽然总是被批驳,但有时候值得参考。对于wait time你要了解什么是wait event,了解常见的wait event。idle的event,non-idle的event,到底先定位哪个问题?idle的event是不是真的就可以忽略? 都是需要商榷的。对于service time你要关心CPU的使用,你的系统花费了多少CPU,都用在哪里?怎么获取这些信息?你是否能够v$sysstat和v$sesstat找到你需要 的信息?

了解常用的调优工具,explain plan, sql trace, 10046 event, 10053 event, other events,tkprof。。。。。。还有最常用的statspack,给你一个statspack report,你能挖掘多少信息?了解常见的数据字典并能够从中迅速获取信息。v$latch, v$lock, v$session, v$session_longops, v$session_wait, v$sysstat.........

熟悉sql优化,这恐怕是个无止境的topic,总之尽力而为吧。。。不停的学习体会,了解各种Hint,一点一点的了解神秘的CBO optimier,了解各种相关的初始化参数,了解典型的SQL计划,nested loop, hash join, merge join, inlist, index range scan, fast full index scan, full table scan......

对sql的优化当然也包括所谓的physical design, 建什么样的索引,什么样的表,了解heap table, IOT, cluster table, btree index, bitmap index,分区表/索引,物化视图......各有什么特点,优点,缺点。

要有大局观,什么时候从session级别着手,什么时候从系统级别着手,什么时候能够斩钉截铁的说:“这不是数据库的问题!请检查硬件/网络/应用。。。”

要有前瞻性,不是一定要等到问题发生的时候才去研究问题,如果用户的并发数翻一倍,系统的压力会增加多少个precent?如果系统的CPU使用率从30%涨到60%,性能的下降将是线性的还是指数性的?下降多少个precent?queuing theory是什么?

最后怎么把所有的这些知识融会贯通?

当然有时候也可以唯心一把,祈祷一下你会获得更多的奇思妙想,更多的灵感。不过有时候你可能会发现,当你绞尽脑汁,百思不得其解,即使在睡梦中也难以释怀之时,灵感却在第二天的清晨不期而至

Sunday, April 20, 2008

10g Time Model

Time model statistics show the amount of CPU time that has been required to complete each type of database processing work. Examples include sql execute elapsed time, parse time elapsed and PL/SQL execution elapsed time statistics.

The most important time model statistic is DB time, which represents the total time spent by Oracle to process all database calls. In fact, it describes the total database workload. DB time is calculated by aggregating the CPU and all non-idle wait times for all sessions in the database after the last startup.

Because DB_TIME is an aggregate value gathered for all non-idle sessions, the total time will nearly always exceed the total elapsed time since instance startup.

For example, an
instance that has been up for 10 hours may have had 60 sessions that were active for 30 minutes each. These would show a total time of 60 × 30 minutes, or 30 hours.

Oracle Database 10g introduces time models for identifying the time spent in various places. The overall system time spent is recorded in the viewV$SYS_TIME_MODEL. Here is the query and its output:
SQL> select * from v$sys_time_model;

This view shows the overall system times as well; however, you may be interested in a more granular view: the session level times. The timing stats are captured at the session level as well, as shown in the viewV$SESS_TIME_MODEL, where all the stats of the current connected sessions, both active and inactive, are visible. The additional column SID specifies the SID of the sessions for which the stats are shown:
SQL> select * from v$sess_time_model where sid=335;


The challenge ahead is how to drill down the interval to same as snapshot's . ! ? This is especially good to certain critical period while cyclical critical job runs.

Saturday, April 19, 2008

Managing Volatile Object Statistics

Volatile objects are objects that can drastically change in size over the
course of a day. For example, tables that are the target of a bulk-load operation (where the
number of rows increases by 10 percent or more) would be considered volatile. Also, tables
that are truncated or dropped, and then rebuilt, would also be considered volatile. Volatile
objects run the risk of having no statistics (if they were dropped and rebuilt) or having inaccurate
statistics.

As part of Oracle’s query optimization, any table with no statistics will have them generated dynamically via the dynamic sampling feature. This “just-in-time” statistics generation ensures that no query will be executed without statistics. The parameter OPTIMIZER_DYNAMIC_SAMPLING needs to be set to a value of 2 (the default) or higher to enable this feature.


SQL> select LAST_ANALYZED,avg_row_len from dba_tables where table_name='T1' and owner='HR';

LAST_ANAL AVG_ROW_LEN
--------- -----------
14-APR-08 3

SQL> select LAST_ANALYZED,avg_row_len,num_rows from dba_tables where table_name='T1' and owner='HR';

LAST_ANAL AVG_ROW_LEN NUM_ROWS
--------- ----------- ----------
14-APR-08 3 1

--To set statistics for a table to NULL, delete and lock the statistics as shown:
SQL> exec dbms_stats.delete_table_stats('HR','T1');

PL/SQL procedure successfully completed.

SQL> select LAST_ANALYZED,avg_row_len,num_rows from dba_tables where table_name='T1' and owner='HR';

LAST_ANAL AVG_ROW_LEN NUM_ROWS
--------- ----------- ----------

--The second option is to set statistics to values that are typical for the table and lock them. To
--achieve this, gather statistics when the table is at a typical size. When complete, lock the table’s
--statistics, as shown in the preceding example.
SQL> exec dbms_stats.lock_table_stats('HR','T1');

PL/SQL procedure successfully completed.

SQL> select LAST_ANALYZED,avg_row_len,num_rows from dba_tables where table_name='T1' and owner='HR';

LAST_ANAL AVG_ROW_LEN NUM_ROWS
--------- ----------- ----------


SQL> insert into hr.t1 values(100);

1 row created.

SQL> commit;

Commit complete.

SQL> analyze table hr.t1 compute statistics;
analyze table hr.t1 compute statistics
*
ERROR at line 1:
ORA-38029: object statistics are locked


SQL> exec dbms_stats.unlock_table_stats('HR','T1');

PL/SQL procedure successfully completed.

SQL> analyze table hr.t1 compute statistics;

Table analyzed.

SQL> select LAST_ANALYZED,avg_row_len,num_rows from dba_tables where table_name='T1' and owner='HR';

LAST_ANAL AVG_ROW_LEN NUM_ROWS
--------- ----------- ----------
18-APR-08 6 2

Friday, June 15, 2007

pga_aggregate_target tuning

"最近再深入了解调优方面的东西
做了一个试验。
有一个表,30个g的数据,在上面建索引。
一开始,pga_aggregate_target设置为200m,发现v$sql_workarea_histogram中有one pass的值,但是没怎么在意后来将设置为1000m,发现速度快了,现在正在做设置为2g的测试,看是否会更快。

我现在就想看看对内存敏感的sql statement操作时,workarea的大小对速度究竟有多大的影响?"

The relevant table is v$sql_workarea_histogram

Thursday, June 14, 2007

about fast_start_mttr_target

Other than reduce time needed for recovery, it also affects performance, shorten checkpoint time. Especially when the DB has batch job running during certain period.