Tuesday, November 27, 2012

All About Statistics In Oracle


In this post I'll try to summarize all sorts of statistics in Oracle along with [How to collect, backup and restore all types of database statistics]. I would recommend to read the full article, as it contains information you may find it valuable in understanding Oracle statistics.



#####################################
Database | Schema | Table | Index Statistics
#####################################

Gather Database Statistics:
=======================
SQL> EXEC DBMS_STATS.GATHER_DATABASE_STATS(
     ESTIMATE_PERCENT=>100,METHOD_OPT=>'FOR ALL COLUMNS SIZE SKEWONLY',
    CASCADE => TRUE,
    degree => DBMS_STATS.AUTO_DEGREE,
    GATHER_SYS => TRUE
    );

CASCADE => TRUE : Gather statistics on the indexes as well. If not used Oracle will decide whether to collect index statistics or not.
DEGREE => The Degree of parallelism (set it as "DBMS_STATS.AUTO_DEGREE" to let Oracle decide the best parallelism degree, or specify a number instead).

options: 
       =>'GATHER' : Gathers statistics on all objects in the schema.
       =>'GATHER AUTO' : Oracle determines which objects need new statistics, and determines how to gather those statistics.
       =>'GATHER STALE': Gathers statistics on stale objects. will return a list of stale objects.
       =>'GATHER EMPTY': Gathers statistics on objects have no statistics. will return a list of no stats objects.
        =>'LIST AUTO' : Returns a list of objects to be processed with GATHER AUTO.
        =>'LIST STALE': Returns a list of stale objects as determined by looking at the *_tab_modifications views.
        =>'LIST EMPTY': Returns a list of objects which currently have no statistics.
GATHER_SYS => TRUE : Gathers statistics on the objects owned by the 'SYS' user.
STATTAB => PROD_STATS :Table will save the current statistics. see SAVE & IMPORT STATISTICS section -last third in this post-.

Note: All the above parameters are valid for all kind of statistics (schema, table,..) except Gather_SYS.
Note: Skew data means the data inside a column is not uniform, there is a particular one or more value are being repeated much than other values in the same column, for example the gender column in employee table with two values (male/female), in a construction or security service company, where most of the employees are male workforce, the gender column in employee table is likely to be skewed but in an entity like a hospital where the number of males almost equal the number of female workforce, the gender column is likely to be not skewed.

What's new?
ESTIMATE_PERCENT=>DBMS_STATS.AUTO_SAMPLE_SIZE => Let Oracle estimate skewed values always gives excellent results.(DEFAULT).
Removed "METHOD_OPT=>'FOR ALL COLUMNS SIZE SKEWONLY'" => As histograms are not recommended to be gathered on all columns.
Removed  "cascade => TRUE" To let Oracle determine whether index statistics to be collected or not.
DEGREE => The Degree of parallelism (set it as "DBMS_STATS.AUTO_DEGREE" to let Oracle decide the best parallelism degree, or specify a number instead).


For faster execution: let Oracle decide the optimal parallelism degree using DBMS_STATS.AUTO_DEGREE

SQL> BEGIN
DBMS_STATS.GATHER_DATABASE_STATS(
ESTIMATE_PERCENT=>DBMS_STATS.AUTO_SAMPLE_SIZE,
degree => DBMS_STATS.AUTO_DEGREE
);
END;
/
 
To Gather STALE statistics ONLY:
 
STALE statistics are the ones that don't represent the current data or simply they are not up to date and may mislead the optimizer when choosing the best execution plans.
 
To query all tables with STALE statistics in the DB:

SQL> select owner,table_name from dba_tab_statistics where stale_stats='YES' order by 1;
 
To gather statistics on tables with STALE statisitcs ONLY execute this command with OPTIONS => 'GATHER STALE':

SQL>  
BEGIN
DBMS_STATS.GATHER_DATABASE_STATS(
ESTIMATE_PERCENT=>DBMS_STATS.AUTO_SAMPLE_SIZE,
CASCADE => TRUE,
DEGREE => DBMS_STATS.AUTO_DEGREE,
OPTIONS => 'GATHER STALE'
);
END;
/

 

Starting from Oracle 10g, Oracle introduced an automated task gathers statistics on all objects in the database that having [stale or missing] statistics, To check the status of that task:
SQL> select status from dba_autotask_client where client_name = 'auto optimizer stats collection';

To Enable Automatic Optimizer Statistics task:
SQL> BEGIN
    DBMS_AUTO_TASK_ADMIN.ENABLE(
    client_name => 'auto optimizer stats collection', 
    operation => NULL, 
    window_name => NULL);
    END;
    /

In case you want to Disable Automatic Optimizer Statistics task:
SQL> BEGIN
    DBMS_AUTO_TASK_ADMIN.DISABLE(
    client_name => 'auto optimizer stats collection', 
    operation => NULL, 
    window_name => NULL);
    END;
    /

To check the tables having stale statistics:

SQL> exec DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO;
SQL> select OWNER,TABLE_NAME,LAST_ANALYZED,STALE_STATS from DBA_TAB_STATISTICS where STALE_STATS='YES';

[update on 03-Sep-2014]
Note: In order to get an accurate information from DBA_TAB_STATISTICS or (*_TAB_MODIFICATIONS, *_TAB_STATISTICS and *_IND_STATISTICS) views, you should manually run DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO procedure to refresh its parent table mon_mods_all$ from SGA recent data, or you have wait for an Oracle internal that refresh that table once a day in 10g onwards [except for 10gR2] or every 15 minutes in 10gR2 or every 3 hours in 9i backward. or when you run manually run one of the GATHER_*_STATS procedures.
[Reference: Oracle Support and MOS ID 1476052.1]

Gather SCHEMA Statistics example:
=============================
SQL> BEGIN
     DBMS_STATS.GATHER_SCHEMA_STATS (
     ownname =>'SCOTT',
     estimate_percent=>DBMS_STATS.AUTO_SAMPLE_SIZE,
     degree=>DBMS_STATS.AUTO_DEGREE,
     cascade=>TRUE);
     END;
     /

CASCADE => TRUE: Gather statistics on the indexes as well. If not used Oracle will determine whether to collect it or not.
DEGREE => The Degree of parallelism (set it as "DBMS_STATS.AUTO_DEGREE" to let Oracle decide the best parallelism degree, or specify a number instead).
ESTIMATE_PERCENT => DBMS_STATS.AUTO_SAMPLE_SIZE : (DEFAULT) Auto set the sample size % for skew(distinct) values (accurate and faster than setting a manual sample size).
METHOD_OPT=>  :  For gathering Histograms:
 FOR COLUMNS SIZE AUTO: You can specify one column between "" instead of all columns.
 FOR ALL COLUMNS SIZE REPEAT: Prevent deletion of histograms and collect it only for columns already have histograms.
 FOR ALL COLUMNS: Collect histograms on all columns.
 FOR ALL COLUMNS SIZE SKEWONLY: Collect histograms for columns have skewed value should test skewness first>.
 FOR ALL INDEXED COLUMNS: Collect histograms for columns have indexes only.


To Gather schema STALE statistics ONLY:
  
SQL>  BEGIN
     DBMS_STATS.GATHER_SCHEMA_STATS (
     ownname =>'SCOTT',
     estimate_percent=>DBMS_STATS.AUTO_SAMPLE_SIZE,
     degree=>DBMS_STATS.AUTO_DEGREE,
     cascade=>TRUE,
     OPTIONS => 'GATHER STALE'
     );
     END;
     /
 
Note: GATHER STALE option can be used with Dictionary, Database and Schema statistics.
 
Gather TABLE Statistics example:
===========================
Check table statistics date:
SQL> select table_name, last_analyzed from user_tables where table_name='T1';

SQL> Begin DBMS_STATS.GATHER_TABLE_STATS (

    ownname => 'SCOTT',
    tabname => 'EMP',
    degree => 2,
    cascade => TRUE,
    METHOD_OPT => 'FOR COLUMNS SIZE AUTO',
    estimate_percent => DBMS_STATS.AUTO_SAMPLE_SIZE);
    END;
    /

Note: Truncating a table will not update table statistics, it will only reset the High Water Mark, you've to re-gather statistics on that table.

Inside "DBA BUNDLE", there is a script called "gather_stats.sh", it will back up the current statistics, gather new statistics on the table/schema you want, and will provide you with the command to restore back the old statistics in case you want to revert back to the old statistics.

To download the latest version of "DBA BUNDLE" please visit this post:
http://dba-tips.blogspot.com/2014/02/oracle-database-administration-scripts.html


Gather Index Statistics:
===================
SQL>

BEGIN
DBMS_STATS.GATHER_INDEX_STATS(ownname => 'SCOTT',indname => 'EMP_I',estimate_percent =>DBMS_STATS.AUTO_SAMPLE_SIZE);
END;
/

####################
Fixed OBJECTS Statistics
####################

What are Fixed objects:
----------------------------
-Fixed objects are the x$ tables (been loaded in SGA during startup) on which V$ views are built (V$SQL etc.).
-If the statistics are not gathered on fixed objects, the Optimizer will use predefined default values for the statistics. These defaults may lead to inaccurate execution plans.
-Statistics on fixed objects are not being gathered automatically or within gathering DB stats.

How frequent to gather stats on fixed objects?
-------------------------------------------------------
Only one time for a representative workload, but it's recommended to gather it whenever you hit one of the following events:

- After a major database or application upgrade.
- After implementing a new module.
- After changing the database configuration. e.g. changing the size of memory pools (sga, pga,..).
- Poor performance/Hang encountered while querying dynamic views e.g. V$ views.

Please note that:
- It's recommended to Gather the fixed object stats during peak hours (whilst the system is busy) or after the peak hours but the sessions are still connected (even if they idle), this will guarantee the population of new statistics representing the actual/average DB load.
- Performance degradation can happen during the gather of fixed objects statistics.
- Having no statistics (or using RULE Based Optimizer against fixed objects) is better than having a non-representative statistics 
on fixed objects.

How to gather stats on fixed objects:
---------------------------------------------

First Check the last analyzed date:
------ -----------------------------------
SQL> select OWNER, TABLE_NAME, LAST_ANALYZED

       from dba_tab_statistics where table_name='X$KGLDP'; 

Second Export the current fixed stats in a table: (in case you need to revert back)
------- -----------------------------------
SQL> EXEC DBMS_STATS.CREATE_STAT_TABLE

       ('OWNER','STATS_TABLE_NAME','TABLESPACE_NAME');

SQL> EXEC dbms_stats.export_fixed_objects_stats

       (stattab=>'STATS_TABLE_NAME',statown=>'OWNER'); 

Third Gather the fixed objects stats:
-------  ------------------------------------
SQL> exec dbms_stats.gather_fixed_objects_stats; 


Note:

New gathered statistics may not immediately take effect unless you flush the shared pool or restart the database.

In case you experienced a bad performance on fixed tables after the new statistics, simply restore back the statistics from the export you have done in step2:

SQL> exec dbms_stats.delete_fixed_objects_stats(); 
SQL> exec DBMS_STATS.import_fixed_objects_stats

       (stattab =>'STATS_TABLE_NAME',STATOWN =>'OWNER');


#################
SYSTEM STATISTICS
#################

What is system statistics:
-------------------------------
System statistics are statistics about CPU speed and IO performance, it enables the CBO to
effectively cost each operation in an execution plan. Introduced in Oracle 9i.

Why gathering system statistics:
----------------------------------------
Oracle highly recommends gathering system statistics during a representative workload,
ideally at peak workload time, in order to provide more accurate CPU/IO cost estimates to the optimizer.
You only have to gather system statistics once.

There are two types of system statistics (NOWORKLOAD statistics & WORKLOAD statistics):

NOWORKLOAD statistics:
-----------------------------------
This will simulate a workload -not the real one but a simulation- and will not collect full statistics, it's less accurate than "WORKLOAD statistics" but if you can't capture the statistics during a typical workload you can use no-workload statistics.
To gather noworkload statistics:
SQL> execute dbms_stats.gather_system_stats(); 


WORKLOAD statistics:
-------------------------------
This will gather statistics during the current workload [which supposed to be representative of the actual system I/O and CPU workload on the DB].
To gather WORKLOAD statistics:
SQL> execute dbms_stats.gather_system_stats('start');
Once the workload window ends after 1,2,3.. hours or whatever, stop the system statistics gathering:
SQL> execute dbms_stats.gather_system_stats('stop');
You can use time interval (minutes) instead of issuing start/stop command manually:
SQL> execute dbms_stats.gather_system_stats('interval',60); 


Check the system values collected:
-------------------------------------------
col pname format a20
col pval2 format a40
select * from sys.aux_stats$;
 


cpuspeedNW:  Shows the noworkload CPU speed, (average number of CPU cycles per second).
ioseektim:    The sum of seek time, latency time, and OS overhead time.
iotfrspeed:  I/O transfer speed, tells optimizer how fast the DB can read data in a single read request.
cpuspeed:      Stands for CPU speed during a workload statistics collection.
maxthr:          The maximum I/O throughput.
slavethr:      Average parallel slave I/O throughput.
sreadtim:     The Single Block Read Time statistic shows the average time for a random single block read.
mreadtim:     The average time (seconds) for a sequential multiblock read.
mbrc:             The average multiblock read count in blocks.

Notes:

-When gathering NOWORKLOAD statistics it will gather (cpuspeedNW, ioseektim, iotfrspeed) system statistics only.
-Above values can be modified manually using DBMS_STATS.SET_SYSTEM_STATS procedure.
-According to Oracle, collecting workload statistics doesn't impose an additional overhead on your system.

Delete system statistics:
------------------------------
SQL> execute dbms_stats.delete_system_stats();


####################
Data Dictionary Statistics
####################

Facts:
-------
> Dictionary tables are the tables owned by SYS and residing in the system tablespace.
> Normally data dictionary statistics in 9i is not required unless performance issues are detected.
> In 10g Statistics on the dictionary tables will be maintained via the automatic statistics gathering job run during the nightly maintenance window.

If you choose to switch off that job for application schema consider leaving it on for the dictionary tables. You can do this by changing the value of AUTOSTATS_TARGET from AUTO to ORACLE using the procedure:

SQL> Exec DBMS_STATS.SET_PARAM(AUTOSTATS_TARGET,'ORACLE');  


When to gather Dictionary statistics:
---------------------------------------------
-After DB upgrades.
-After the creation of a new big schema.
-Before and after big datapump operations.

Check last Dictionary statistics date:
---------------------------------------------
SQL> select table_name, last_analyzed from dba_tables

     where owner='SYS' and table_name like '%$' order by 2; 

Gather Dictionary Statistics:  
-----------------------------------
SQL> EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;

->Will gather stats on 20% of SYS schema tables.
or...
SQL> EXEC DBMS_STATS.GATHER_SCHEMA_STATS ('SYS');

->Will gather stats on 100% of SYS schema tables.
or...
SQL> EXEC DBMS_STATS.GATHER_DATABASE_STATS
(gather_sys=>TRUE);
->Will gather stats on the whole DB+SYS schema.



################
Extended Statistics "11g onwards"
################

Extended statistics can be gathered on columns based on functions or column groups.

Gather extended stats on column function:
====================================
If you run a query having in the WHERE statement a function like upper/lower the optimizer will be off and index on that column will not be used:
SQL> select count(*) from EMP where lower(ename) = 'scott'; 


In order to make optimizer work with function-based terms you need to gather extended stats:

1-Create extended stats:
>>>>>>>>>>>>>>>>>>>>
SQL> select dbms_stats.create_extended_stats
('SCOTT','EMP','(lower(ENAME))') from dual;

2-Gather histograms:
>>>>>>>>>>>>>>>>>
SQL> exec dbms_stats.gather_table_stats
('SCOTT','EMP', method_opt=> 'for all columns size skewonly');

OR
----

*You can do it also in one Step:
>>>>>>>>>>>>>>>>>>>>>>>>>

SQL> Begin dbms_stats.gather_table_stats

     (ownname => 'SCOTT',tabname => 'EMP',
     method_opt => 'for all columns size skewonly for
     columns (lower(ENAME))');
     end;
     /

To check the Existence of extended statistics on a table:
----------------------------------------------------------------------
SQL> select extension_name,extension from dba_stat_extensions 
where owner='SCOTT'and table_name = 'EMP';
SYS_STU2JLSDWQAFJHQST7$QK81_YB (LOWER("ENAME"))

Drop extended stats on column function:
------------------------------------------------------
SQL> exec dbms_stats.drop_extended_stats
('SCOTT','EMP','(LOWER("ENAME"))');

Gather extended stats on column group: -related columns-
=================================
Certain columns in a table that are part of a join condition (where statements are correlated e.g.(country, state). You want to make the optimizer aware of this relationship between two columns and more instead of using separate statistics for each column. By creating extended statistics on a group of columns, the Optimizer can determine a more accurate the relation between the columns are used together in a where clause of a SQL statement. e.g. columns like country_id and state_name the have a relationship, state like Texas can only be found in the USA so the value of state_name is always influenced by country_id.
If there are extra columns are referenced in the "WHERE statement with the column group the optimizer will make use of column group statistics.

1- create a column group:
>>>>>>>>>>>>>>>>>>>>>
SQL> select dbms_stats.create_extended_stats
('SH','CUSTOMERS', '(country_id,cust_state_province)')from dual;
2- Re-gather stats|histograms for table so optimizer can use the newly generated extended statistics:
>>>>>>>>>>>>>>>>>>>>>>>
SQL> exec dbms_stats.gather_table_stats ('SH','customers',
method_opt=> 'for all columns size skewonly');

OR
---


*You can do it also in one Step:
>>>>>>>>>>>>>>>>>>>>>>>>>

SQL> Begin dbms_stats.gather_table_stats

     (ownname => 'SH',tabname => 'CUSTOMERS',
     method_opt => 'for all columns size skewonly for
     columns (country_id,cust_state_province)');
     end; 
     /

Drop extended stats on column group:
--------------------------------------------------
SQL> exec dbms_stats.drop_extended_stats
('SH','CUSTOMERS', '(country_id,cust_state_province)');


#########
Histograms
#########

What are Histograms?

-----------------------------
> Holds data about values within a column in a table for the number of occurrences for a specific value/range.
> Used by CBO to optimize a query to use whatever index Fast Full scan or table full scan.
> Usually being used against columns have data being repeated frequently like country or city column.
> gathering histograms on a column having distinct values (PK) is useless because values are not repeated.
> Two types of Histograms can be gathered:
  -Frequency histograms are when distinct values (buckets) in the column is less than 255 
(e.g. the number of countries is always less than 254).
  -Height balanced histograms: are similar to frequency histograms in their design, but distinct values bigger than 254.
> Collected by DBMS_STATS (which by default doesn't collect histograms, it deletes them if you didn't use the parameter).
> Mainly being gathered on foreign key columns/columns in WHERE statement.
> Help in SQL multi-table joins.
> Column histograms like statistics are being stored in the data dictionary.
> If the application is exclusive uses bind variables, Oracle recommends deleting any existing 
histograms and disabling Oracle histograms generation.

Cautions:
   – Do not create histograms on Columns that are not being queried.
   – Do not create histograms on every column of every table.
   – Do not create histograms on the primary key column of a table.

Verify the existence of histograms:
---------------------------------------------
SQL> select column_name,histogram from dba_tab_col_statistics

     where owner='SCOTT' and table_name='EMP'; 

Creating Histograms:
---------------------------
e.g.

SQL> Exec dbms_stats.gather_schema_stats
     (ownname => 'SCOTT',
     estimate_percent => dbms_stats.auto_sample_size,
     method_opt => 'for all columns size auto',
     degree => 7);


method_opt:
FOR COLUMNS SIZE AUTO                 => Fastest. you can specify one column instead of all columns.
FOR ALL COLUMNS SIZE REPEAT     => Prevent deletion of histograms and collect it only 
for columns already have histograms.
FOR ALL COLUMNS => collect histograms on all columns.
FOR ALL COLUMNS SIZE SKEWONLY => collect histograms for columns have skewed value.
FOR ALL INDEXES COLUMNS      => collect histograms for columns have indexes.

Note: AUTO & SKEWONLY will let Oracle decide whether to create the Histograms or not.

Check the existence of Histograms:
SQL> select column_name, count(*) from dba_tab_histograms

     where OWNER='SCOTT' table_name='EMP' group by column_name; 

Drop Histograms: 11g
----------------------
e.g.
SQL> Exec dbms_stats.delete_column_stats

     (ownname=>'SH', tabname=>'SALES',
     colname=>'PROD_ID', col_stat_type=> HISTOGRAM);


Stop gather Histograms: 11g
------------------------------
[This will change the default table options]
e.g.
SQL> Exec dbms_stats.set_table_prefs

     ('SH', 'SALES','METHOD_OPT', 'FOR ALL COLUMNS SIZE AUTO,FOR COLUMNS SIZE 1 PROD_ID');
>Will continue to collect histograms as usual on all columns in the SALES table except for PROD_ID column.

Drop Histograms: 10g
----------------------
e.g.
SQL> exec dbms_stats.delete_column_stats
(user,'T','USERNAME');


################################
Save/IMPORT & RESTORE STATISTICS:
################################
====================
Export /Import Statistics:
====================
In this way statistics will be exported (backed up) into a table then imported (restored) later whenever needed from that table.

1-Create STATS TABLE:
-  -----------------------------
SQL> Exec dbms_stats.create_stat_table
(ownname => 'SYSTEM', stattab => 'prod_stats',tblspace => 'USERS'); 

2-Export/Backup statistics to the STATS table: [Backup Statistics]
-------------------------------------------------------

In Oracle you have the flexability to backup & restore the statistics from Database level up to column level!

The following will back up the statistics into PROD_STATS table which we just created under SYSTEM schema in Step 1.


Backup Database statistics:
SQL> Exec dbms_stats.export_database_stats
(statown => 'SYSTEM', stattab => 'prod_stats');
Backup System statistics:
SQL> Exec dbms_stats.export_SYSTEM_stats
(statown => 'SYSTEM', stattab => 'prod_stats');
Backup Dictionary statistics:
SQL> Exec dbms_stats.export_Dictionary_stats
(statown => 'SYSTEM', stattab => 'prod_stats');
Backup Fixed Tables statistics:
SQL> Exec dbms_stats.export_FIXED_OBJECTS_stats
(statown => 'SYSTEM', stattab => 'prod_stats');
Backup Schema statistics:
SQL> EXEC DBMS_STATS.EXPORT_SCHEMA_STATS
(ownname=>'SCHEMA_NAME',stattab=>'STATS_TABLE',statown=>'STATS_TABLE_OWNER');
[e.g. Backup statistics of SCOTT SCHEMA]
SQL> EXEC DBMS_STATS.EXPORT_SCHEMA_STATS(ownname=>'SCOTT',stattab=>'prod_stats',statown=>'system');
Backup Table statistics:
[e.g. Backup statistics of SCOTT.EMP Table]
SQL> Conn scott/tiger
SQL> Exec dbms_stats.export_TABLE_stats
(ownname => 'SCOTT',tabname => 'EMP',statown => 'SYSTEM', stattab => 'prod_stats');
Backup Index statistics:
SQL> Exec dbms_stats.export_INDEX_stats
(ownname => 'SCOTT',indname => 'PK_EMP',statown => 'SYSTEM', stattab => 'prod_stats');
Backup Column statistics:
SQL> Exec dbms_stats.export_COLUMN_stats 
(ownname=>'SCOTT',tabname=>'EMP',colname=>'EMPNO',statown => 'SYSTEM', stattab=>'prod_stats');

Parameters:
ownname: The owner of the object that will have its statistics backed up.
tabname: The table name which will have its stats backed up.
indname: The index name which will have its stats backed up.
statown: The owner of the table which stores the backed up statistics.
stattab: The table which stores the backed up statistics.

3-Import/Restore statistics from PROD_STATS table to the dictionary:
--------------------------------------------------------------------------------------
As agreed above we already created a table called "
prod_stats" under schema "SYSTEM" to hold the statistics in step1, then we loaded it with statistics we want to backup in step2.

Restore Database statistics:
SQL> Exec DBMS_STATS.IMPORT_DATABASE_STATS

     (stattab => 'prod_stats',statown => 'SYSTEM');

Restore System statistics:
SQL> Exec DBMS_STATS.IMPORT_SYSTEM_STATS

     (stattab => 'prod_stats',statown => 'SYSTEM');

Restore
Dictionary statistics:
SQL> Exec DBMS_STATS.IMPORT_Dictionary_STATS

     (stattab => 'prod_stats',statown => 'SYSTEM');

Restore
Fixed Tables statistics:
SQL> Exec DBMS_STATS.IMPORT_FIXED_OBJECTS_STATS

     (stattab => 'prod_stats',statown => 'SYSTEM');

Restore
Schema statistics:
If the schema statistics is locked, unlock it first: exec dbms_stats.unlock_schema_stats('SCOTT');

SQL> Exec DBMS_STATS.IMPORT_SCHEMA_STATS

     (ownname => 'SCOTT',stattab => 'prod_stats', statown => 'SYSTEM');

Restore
Table and its indexes statistics:
If the table statistics is locked, unlock it first: exec dbms_stats.unlock_table_stats('SCOTT','EMP');

SQL> Exec dbms_stats.import_TABLE_stats
     ( ownname => 'SCOTT', stattab => 'prod_stats',tabname => 'EMP');

Restore
Index statistics:
SQL> Exec dbms_stats.import_INDEX_stats

     ( ownname => 'SCOTT', stattab => 'prod_stats', indname => 'PK_EMP');

Restore
COLUMN statistics:
SQL> Exec dbms_stats.import_COLUMN_stats

     (ownname=>'SCOTT',tabname=>'EMP',colname=>'EMPNO',stattab=>'prod_stats');

Parameters:
ownname: The owner of the object that will have its statistics backed up.
tabname: The table name which will have its stats backed up.
indname: The index name which will have its stats backed up.
statown: The owner of the table which stores the backed up statistics.
stattab: The table which stores the backed up statistics.

4-Drop the table holding the STATISTICS:
---------------------------------------------------
SQL> Exec dbms_stats.DROP_STAT_TABLE 
(stattab => 'prod_stats',ownname => 'SYSTEM');

===========
=========
Restore Database Statistics: -From Dictionary-
===============
=====
Old statistics are saved automatically in SYSAUX for 31 day.

Restore Dictionary stats as of timestamp:
------------------------------------------------------
SQL> Exec DBMS_STATS.RESTORE_DICTIONARY_STATS(sysdate-1); 


Restore Database stats as of timestamp:
----------------------------------------------------
SQL> Exec DBMS_STATS.RESTORE_DATABASE_STATS(sysdate-1); 


Restore SYSTEM stats as of timestamp:
----------------------------------------------------
SQL> Exec DBMS_STATS.RESTORE_SYSTEM_STATS(sysdate-1); 


Restore FIXED OBJECTS stats as of timestamp:
----------------------------------------------------------------
SQL> Exec DBMS_STATS.RESTORE_FIXED_OBJECTS_STATS(sysdate-1); 


Restore SCHEMA stats as of timestamp:
---------------------------------------
[e.g. Restore statistics of  SCOTT schema]
If the schema statistics is locked, unlock it first: exec dbms_stats.unlock_schema_stats('SCOTT');

SQL> Exec dbms_stats.restore_SCHEMA_stats
     (ownname=>'SCOTT',AS_OF_TIMESTAMP=>sysdate-1); 
OR:
SQL> Exec dbms_stats.restore_schema_stats

     (ownname=>'SCOTT',AS_OF_TIMESTAMP=>'20-JUL-2008 11:15:00AM');

Restore Table stats as of timestamp:
------------------------------------------------
[e.g. Restore statistics of  SCOTT.EMP Table]
If the table statistics is locked, unlock it first: exec dbms_stats.unlock_table_stats('SCOTT','EMP');

SQL> Exec DBMS_STATS.RESTORE_TABLE_STATS
     (ownname=>'SCOTT', tabname=>'EMP',AS_OF_TIMESTAMP=>sysdate-1);


Delete Statistics:
==============
For Database stats:
SQL> Exec DBMS_STATS.DELETE_DATABASE_STATS ();
For System stats:
SQL> Exec DBMS_STATS.DELETE_SYSTEM_STATS ();
For Dictionary stats:
SQL> Exec DBMS_STATS.DELETE_DICTIONARY_STATS ();
For Fixed Tables stats:
SQL> Exec DBMS_STATS.DELETE_FIXED_OBJECTS_STATS ();
For Schema stats:
SQL> Exec DBMS_STATS.DELETE_SCHEMA_STATS ('SCOTT');
For Table stats and it's indexes:
SQL> Exec dbms_stats.DELETE_TABLE_stats
(ownname=>'SCOTT',tabname=>'EMP');
For Index:
SQL> Exec dbms_stats.DELETE_INDEX_stats
(ownname => 'SCOTT',indname => 'PK_EMP');
For Column:
SQL> Exec dbms_stats.DELETE_COLUMN_stats
(ownname =>'SCOTT',tabname=>'EMP',colname=>'EMPNO');

Note: This procedure can be rollback by restoring STATS using DBMS_STATS.RESTORE_ procedure mentioned in "
Restore Database Statistics" section.


Pending Statistics:  "11g onwards"
===============

What is Pending Statistics:
Pending statistics is a feature let you test the newly gathered statistics without letting the CBO (Cost Based Optimizer) use them "system-wide" unless you publish them.

How to use Pending Statistics:

Switch ON pending statistics mode:
[Activiate Pending Statistics mode Globally]
SQL> Exec DBMS_STATS.SET_GLOBAL_PREFS('PUBLISH','FALSE');
Note: Any new statistics will be gathered on the database will be marked PENDING unless you change back the previous parameter to true as shown at the end of this paragraph.
[Activiate Pending Statistics mode Specifically, e.g. for sh.sales table]

SQL> Exec DBMS_STATS.SET_TABLE_PREFS('sh','sales','publish','false');

Gather statistics: "as you used to do"
SQL> Exec DBMS_STATS.GATHER_TABLE_STATS('sh','SALES');
Enable the use of pending statistics on your session ONLY to know how it performs:
SQL> alter session set optimizer_use_pending_statistics=TRUE;

Then any SQL statement you will run will use the new pending statistics...

When proven OK, PUBLISH the pending statistics:

[For a Specific Table]
SQL> Exec DBMS_STATS.PUBLISH_PENDING_STATS('sh','SALES');
[For All StatisticsTable]
SQL> Exec DBMS_STATS.PUBLISH_PENDING_STATS(); 

If NOT proven OK, DELETE the pending statistics:
[For a Specific Table]
SQL> Exec DBMS_STATS.DELETE_PENDING_STATS('sh','SALES');
[For All StatisticsTable]
SQL> Exec DBMS_STATS.DELETE_PENDING_STATS();

Once you finish don't forget to DEACTIVATE the Pending Statistics mode:
[De-activiate Pending Statistics mode Globally]
SQL> Exec DBMS_STATS.SET_GLOBAL_PREFS('PUBLISH','TRUE');
[De-activiate Pending Statistics mode Specifically, e.g. for SH.SALES table]


SQL> Exec DBMS_STATS.SET_TABLE_PREFS('sh','sales','publish','true');

>If you forgot the last step, all newly gathered statistics on the database will be marked as PENDING, the thing may put you and your DBA colleagues in a pulling hair situation :-)


Lock Statistics:
=============

Gathering new statistics is not always a good approach, this may change your applications queries'/reports' execution plans to the worst, it's not guaranteed that gathering new statistics will lead to better execution plans! .I've learned this lesson before in a hard way! but having a backup of the old statistics before gathering new ones has saved my day!. 
This is why you want to avoid having such scenario, where one of the DBA's in your team has accidentally gathered new statistics on the whole DB ـــscrambling most of the execution plans of application queries, in the hope of generating better execution plans. In this case, you need to lock the statistics of one or more schema or on key tables in order to prevent their statistics from being refreshed by such unattended maintenance activities.

To lock the statistics on all tables under a specific schema:
SQL> exec dbms_stats.lock_schema_stats('SCHEMA_NAME');
e.g. exec dbms_stats.lock_schema_stats('SCOTT');

To lock the statistics on a specific table:
SQL> exec dbms_stats.lock_table_stats('OWNER','TABLE_NAME'');
e.g. exec dbms_stats.lock_table_stats('SCOTT','EMP');

Note: This will lock the table's statistics and its indexes.

When you have a need to gather new statistics on those tables that having their statistics locked, you need first to unlock the statistics then gather new statistics as usual.

To check all tables that have their statistics locked:
SQL> select OWNER, TABLE_NAME, LAST_ANALYZED, STATTYPE_LOCKED from DBA_TAB_STATISTICS
where STATTYPE_LOCKED is not null
and OWNER not in ('SYS','SYSTEM','SQLTXPLAIN','WMSYS')
order by OWNER, TABLE_NAME; 

To unlock all tables under a specific schema:
SQL> exec dbms_stats.unlock_schema_stats('SCHEMA_NAME');
e.g. exec dbms_stats.unlock_schema_stats('SCOTT');

To unlock a specific table:
SQL> exec dbms_stats.unlock_table_stats('OWNER','TABLE_NAME'');
e.g. exec dbms_stats.unlock_table_stats('SCOTT','EMP');

Note: This will unlock the table's statistics and its indexes.

=========
Advanced:
=========

To Check current Statistics history retention period (days):
-------------------------------------------------------------------
SQL> select dbms_stats.get_stats_history_retention from dual;
SQL> select dbms_stats.get_stats_history_availability 
from dual;
To modify current Statistics history retention period (days):
-------------------------------------------------------------------
SQL> Exec dbms_stats.alter_stats_history_retention(60); 


Purge statistics older than 10 days:
------------------------------------------
SQL> Exec DBMS_STATS.PURGE_STATS(SYSDATE-10);

Procedure To claim space after purging statstics:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Space will not be claimed automatically when you purge stats, you must claim it manually using this procedure:

Check Stats tables size:
>>>>>>
col Mb form 9,999,999

col SEGMENT_NAME form a40
col SEGMENT_TYPE form a6
set lines 160
select sum(bytes/1024/1024) Mb,
segment_name,segment_type from dba_segments
where  tablespace_name = 'SYSAUX'
and segment_name like 'WRI$_OPTSTAT%'
and segment_type='TABLE'
group by segment_name,segment_type
order by 1 asc
/

Check Stats indexes size:
>>>>>

col Mb form 9,999,999
col SEGMENT_NAME form a40
col SEGMENT_TYPE form a6
set lines 160
select sum(bytes/1024/1024) Mb, segment_name,segment_type
from dba_segments
where  tablespace_name = 'SYSAUX'
and segment_name like '%OPT%'
and segment_type='INDEX'
group by segment_name,segment_type
order by 1 asc
/

Move Stats tables in same tablespace:
>>>>>

select 'alter table '||segment_name||'  move tablespace
SYSAUX;' from dba_segments
where tablespace_name = 'SYSAUX'
and segment_name like '%OPT%'
and segment_type='TABLE'
/

Rebuild stats indexes:
>>>>>>

select 'alter index '||segment_name||'  rebuild online;'
from dba_segments
where tablespace_name = 'SYSAUX'
and segment_name like '%OPT%'
and segment_type='INDEX'
/

Check for un-usable indexes:
>>>>>

select  di.index_name,di.index_type,di.status
from dba_indexes di , dba_tables dt
where  di.tablespace_name = 'SYSAUX'
and dt.table_name = di.table_name
and di.table_name like '%OPT%'
order by 1 asc


Changing the optimizer behaviour by manipulating the number of rows of a table:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
To show a table bigger than it's actuall size in order to make the optimizer favors INDEX SCANS plans over TABLE SCANS.
e.g. Manuipluating the number of rows statistic for table SH.SALES:
exec dbms_stats.set_table_stats('sh','sales',numrows => 9000000, numblks => 19000 );

References:
http://docs.oracle.com/cd/E18283_01/appdev.112/e16760/d_stats.htm


16 comments:

  1. Well put together. Nice summary, Thanks.

    SF.

    ReplyDelete
  2. This is a pretty summarized entry for stats.

    Thank you

    ReplyDelete
  3. This task became an automated task starting from 10g, To check the status of that task:
    SQL> select * from dba_autotask_client where client_name = "auto optimizer stats collection" ;


    does this mean that we no need to gather stats from 10g onwards

    ReplyDelete
  4. As per Oracle documentation, this task gathers statistics on all objects in the database which have stale or missing statistics.

    Also you can:

    Enable automatic optimizer statistics:
    SQL> BEGIN
    DBMS_AUTO_TASK_ADMIN.ENABLE(
    client_name => 'auto optimizer stats collection',
    operation => NULL,
    window_name => NULL);
    END;
    /

    Disable automatic optimizer statistics:
    SQL> BEGIN
    DBMS_AUTO_TASK_ADMIN.DISABLE(
    client_name => 'auto optimizer stats collection',
    operation => NULL,
    window_name => NULL);
    END;
    /

    To check the tables that have stale statistics:
    SQL> select OWNER,TABLE_NAME,LAST_ANALYZED,STALE_STATS from DBA_TAB_STATISTICS where STALE_STATS='YES';

    I've updated the post with these information, I recommend you to check this reference:
    http://docs.oracle.com/cd/B28359_01/server.111/b28274/stats.htm#i41282

    And thanks for highlighting this un clear information in the post.

    ReplyDelete
  5. DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO needs to be run before checking the stale_statistics column. Otherwise wrong info is displayed

    ReplyDelete
  6. Thanks a lot for highlighting that issue, your comment is very true, I would like to support it with this excerpt from MOS (Doc ID 1476052.1)

    "When querying *_TAB_MODIFICATIONS view you should ensure that you run DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO before doing so in order to obtain accurate results."

    The same saying applies on DBA_TAB_STATISTICS as both *_TAB_STATISTICS & *_TAB_MODIFICATIONS views are showing their data from mon_mods_all$ table.
    Oracle automatically updates table mon_mods_all$ from SGA recent data every:

    - For Oracle versions older than 10g mon_mods_all$ table is getting refreshed every 3hours (approx).
    - For Oracle version 10gR2 [only] mon_mods_all$ table is getting refreshed every 15 minutes (+1min run time).
    - For Oracle versions 10g onwards [except for 10gR2] mon_mods_all$ table is getting refreshed once a day.

    This table also get updated when you manually execute GATHER_*_STATS procedure which internally calls DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO procedure.

    In case you don't want to wait for the internal job to run, you can manually run DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO procedure to get the table mon_mods_all$ up to date from SGA data.

    Conclusion:
    What you have said is very true. In general, in order to get accurate information from (*_TAB_MODIFICATIONS, *_TAB_STATISTICS and *_IND_STATISTICS) views you have to manually run DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO procedure.

    References:
    - I've opened an SR with Oracle Support to get the above information, unfortunately Oracle's official documentations doesn't enough information about DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO procedure.
    - MOS Doc ID 1476052.1

    I've updated the post with those information, and thank you again for highlighting that issue.

    ReplyDelete
  7. Thanx for the overview. Maybe you can add info about locked statistics to get the info complete.

    Regards,

    Laurens

    ReplyDelete
  8. Thanks so much for sharing...Quite Helpful

    ReplyDelete
  9. Thanks a Million... very comprehensive reference for statistics ...

    ReplyDelete
  10. thanks a million. very helpfull...

    ReplyDelete
  11. Thanks a lot Laurens for highlighting this. Yes locking statistics is an important part I've missed. I've already covered it in the post.

    ReplyDelete
  12. when i run your script against one schema I cannot see any date update on the last_analyzed column it's same before and after:
    select last_analyzed from dba_tables where owner='WOAPIFZ';


    LAST_ANALYZED
    --------------------
    18-Nov-2019 22:00:39
    09-Apr-2018 22:00:08
    09-Apr-2018 22:00:08
    18-Nov-2019 22:02:40
    17-Nov-2015 07:36:48

    ReplyDelete
    Replies
    1. Most probably the statistics are locked on this schema, this is why it will not allow new statistics to be gathered.

      This query will return all the tables that have their statistics locked in your database:
      select OWNER, TABLE_NAME, LAST_ANALYZED, STATTYPE_LOCKED LOCKED from DBA_TAB_STATISTICS
      where STATTYPE_LOCKED is not null
      and OWNER not in ('SYS','SYSTEM','SYSMAN','SQLTXPLAIN','WMSYS')
      order by OWNER, TABLE_NAME;

      To UNLOCK the statistics of a particular schema:
      exec dbms_stats.unlock_schema_stats(<'SCHEMA_NAME>');

      Delete
    2. script updated, the new script can check if statistics is locked, if locked; it will unlock stats first, gather statistic, and last; it will lock statistics back.

      Delete
  13. Very helpful and informative. Looking something like this for PostgreSQL as well.

    ReplyDelete
  14. Great overview about dbms_stats

    ReplyDelete