{"id":33,"date":"2016-10-04T16:36:00","date_gmt":"2016-10-04T16:36:00","guid":{"rendered":"\/colin\/morethanfour\/post\/An-exercise-in-Interval-Partitioning"},"modified":"2016-10-04T16:36:00","modified_gmt":"2016-10-04T16:36:00","slug":"an-exercise-in-interval-partitioning","status":"publish","type":"post","link":"https:\/\/spacefold.com\/colin\/morethanfour\/2016\/10\/04\/an-exercise-in-interval-partitioning\/","title":{"rendered":"An exercise in Interval Partitioning"},"content":{"rendered":"<p>Consider a table, LOG_DATA, with lots of rows:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">select<\/span> trunc( LOG_TIME, <span style=\"color: #006080\">'MM'<\/span> ) <span style=\"color: #0000ff\">as<\/span> <span style=\"color: #0000ff\">MONTH<\/span>, <span style=\"color: #0000ff\">count<\/span>(1) <span style=\"color: #0000ff\">as<\/span> <span style=\"color: #0000ff\">Count<\/span><br><span style=\"color: #0000ff\">from<\/span> Log_Data<br><span style=\"color: #0000ff\">group<\/span> <span style=\"color: #0000ff\">by<\/span> trunc(LOG_TIME,<span style=\"color: #006080\">'MM'<\/span>)<br><span style=\"color: #0000ff\">order<\/span> <span style=\"color: #0000ff\">by<\/span> trunc(LOG_TIME,<span style=\"color: #006080\">'MM'<\/span>);<\/pre>\n<p><\/div>\n<p><img loading=\"lazy\" decoding=\"async\" title=\"image\" style=\"display: inline\" alt=\"image\" src=\"http:\/\/www.spacefold.com\/colin\/morethanfour\/wp-content\/uploads\/image_10.png\" width=\"212\" height=\"173\"><\/p>\n<p>Perhaps this would be better served by a table with RANGE partitioning, using the INTERVAL syntax of 11g to automatically create partitions as data accumulates?<\/p>\n<p>Unfortunately, LOG_TIME is data type timestamp with time zone, and we can&#8217;t create an interval partition on timestamp columns (yet!)<\/p>\n<p>Also, we can&#8217;t use an expression in the interval definition. So, we need a DATE column:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">create<\/span> <span style=\"color: #0000ff\">table<\/span> Log_Data_P<br>(<br>   ID <span style=\"color: #0000ff\">integer<\/span>,<br>   PARENT_ID <span style=\"color: #0000ff\">integer<\/span>,<br>   LOG_TIME <span style=\"color: #0000ff\">timestamp<\/span> <span style=\"color: #0000ff\">with<\/span> <span style=\"color: #0000ff\">time<\/span> <span style=\"color: #0000ff\">zone<\/span> <span style=\"color: #0000ff\">default<\/span> systimestamp,<br>   STACK_LEVEL <span style=\"color: #0000ff\">integer<\/span>,<br>   PACKAGE_NAME varchar2(30),<br>   PROCEDURE_NAME varchar2(30),<br>   LOG_LABEL varchar2(100),<br>   LOG_VALUE varchar2(1000),<br>   LOG_DURATION number,<br>   PROC_DURATION number,<br>   STEP_DURATION number,<br>   LOG_CLOSED <span style=\"color: #0000ff\">integer<\/span>,LOG_DATE <span style=\"color: #0000ff\">date<\/span> <span style=\"color: #0000ff\">default<\/span> sysdate<br>)<br>partition <span style=\"color: #0000ff\">by<\/span> range ( LOG_DATE )<br><span style=\"color: #0000ff\">interval<\/span> ( numtoyminterval( 1, <span style=\"color: #006080\">'MONTH'<\/span> ) )<br>(<br>   partition part_01 <span style=\"color: #0000ff\">values<\/span> <span style=\"color: #0000ff\">less<\/span> <span style=\"color: #0000ff\">than<\/span> ( <span style=\"color: #0000ff\">date<\/span> <span style=\"color: #006080\">'2016-05-01'<\/span> )<br>);<\/pre>\n<p><\/div>\n<p>Here, we have added a LOG_DATE column to the end of the table structure. I&#8217;ve given it a default value and checked the code that writes to the table, to ensure there are no conflicts (there aren&#8217;t). <\/p>\n<p>Now let&#8217;s insert into the new table, with the rows from the original (spanning several months):<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\">insert <span style=\"color: #0000ff\">into<\/span> Log_Data_P<br><span style=\"color: #0000ff\">select<\/span><br>   ID, PARENT_ID, LOG_TIME, STACK_LEVEL, PACKAGE_NAME, PROCEDURE_NAME, LOG_LABEL, LOG_VALUE,<br>   LOG_DURATION, PROC_DURATION, STEP_DURATION, LOG_CLOSED, <span style=\"color: #0000ff\">cast<\/span>( LOG_TIME <span style=\"color: #0000ff\">as<\/span> <span style=\"color: #0000ff\">DATE<\/span> )<br><span style=\"color: #0000ff\">from<\/span> Log_Data;<br><span style=\"color: #0000ff\">commit<\/span>;<\/pre>\n<p><\/div>\n<p>No errors&#8230; let&#8217;s see what partitions we have, at the end of this process:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">select<\/span> PARTITION_NAME, <span style=\"color: #0000ff\">INTERVAL<\/span>, HIGH_VALUE<br>   <span style=\"color: #0000ff\">from<\/span> DBA_Tab_Partitions<br>   <span style=\"color: #0000ff\">where<\/span> TABLE_NAME = <span style=\"color: #006080\">'LOG_DATA_P'<\/span><br>   <span style=\"color: #0000ff\">order<\/span> <span style=\"color: #0000ff\">by<\/span> PARTITION_POSITION;<br> <br>PARTITION_NAME     <span style=\"color: #0000ff\">INTERVAL<\/span>   HIGH_VALUE<br>----------------<span style=\"color: #008000\">-- ---------- -----------------------------------------------------------<\/span><br>PART_01            <span style=\"color: #0000ff\">NO<\/span>         TO_DATE(<span style=\"color: #006080\">' 2016-05-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15277         YES        TO_DATE(<span style=\"color: #006080\">' 2016-06-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15278         YES        TO_DATE(<span style=\"color: #006080\">' 2016-07-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15280         YES        TO_DATE(<span style=\"color: #006080\">' 2016-08-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15281         YES        TO_DATE(<span style=\"color: #006080\">' 2016-09-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15279         YES        TO_DATE(<span style=\"color: #006080\">' 2016-10-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br>SYS_P15282         YES        TO_DATE(<span style=\"color: #006080\">' 2016-11-01 00:00:00'<\/span>, <span style=\"color: #006080\">'SYYYY-MM-DD HH24:MI:SS'<\/span> )<br> <br>7 <span style=\"color: #0000ff\">rows<\/span> selected.<\/pre>\n<p><\/div>\n<p>Interesting&#8230; we have 6 new partitions over and above the original &#8220;PART_01&#8221; partition that we created. Obviously, Oracle has allocated some automatically-generated names to the partitions. <\/p>\n<p>OK, so now the time has come to perform some pruning on our log table. Let&#8217;s delete the oldest partition:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">SQL<\/span>&gt; <span style=\"color: #0000ff\">alter<\/span> <span style=\"color: #0000ff\">table<\/span> Log_Data_P <span style=\"color: #0000ff\">drop<\/span> partition PART_01;<br> <br>ORA-14758: <span style=\"color: #0000ff\">Last<\/span> partition <span style=\"color: #0000ff\">in<\/span> the range <span style=\"color: #0000ff\">section<\/span> cannot be dropped<\/pre>\n<p><\/div>\n<p>Uh-oh. It&#8217;s not as simple as we&#8217;d hoped. More about dropping partitions in this excellent post here:<\/p>\n<ul>\n<li><a href=\"https:\/\/prutser.wordpress.com\/2010\/01\/11\/dropping-interval-partitions\/\">https:\/\/prutser.wordpress.com\/2010\/01\/11\/dropping-interval-partitions\/<\/a><\/li>\n<\/ul>\n<h3>Follow up #1 &#8211; OK, so try something sneaky, and drop the second-to-last partition<\/h3>\n<p>I had an idea. What if we define the table with an artificially small initial partition, with a start date that is earlier than any data in the table? <\/p>\n<p>If we take a look at our Log_Data table today:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">SQL<\/span>&gt; <span style=\"color: #0000ff\">select<\/span> <span style=\"color: #0000ff\">count<\/span>(1), <span style=\"color: #0000ff\">min<\/span>(LOG_TIME) <span style=\"color: #0000ff\">from<\/span> Log_Data;<br> <br>  <span style=\"color: #0000ff\">COUNT<\/span>(1)   <span style=\"color: #0000ff\">MIN<\/span>(LOG_TIME)<br>--------<span style=\"color: #008000\">--   -----------------------------------<\/span><br>   1172660   12-OCT-16 10.57.17.516528 PM -05:00<\/pre>\n<p><\/div>\n<p>1.1 million rows, since October 12. <\/p>\n<p>Now let&#8217;s define the Log_Data_P table again, with a start date of 2016-05-01 as described above, and insert the rows from our Log_Data table. <\/p>\n<p>What do the partitions look like now?<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">select<\/span> PARTITION_NAME, <span style=\"color: #0000ff\">INTERVAL<\/span>, HIGH_VALUE<br>   <span style=\"color: #0000ff\">from<\/span> DBA_Tab_Partitions<br>   <span style=\"color: #0000ff\">where<\/span> TABLE_NAME = <span style=\"color: #006080\">'LOG_DATA_P'<\/span><br>   <span style=\"color: #0000ff\">order<\/span> <span style=\"color: #0000ff\">by<\/span> PARTITION_POSITION;<br> <br>PARTITION_NAME   <span style=\"color: #0000ff\">INTERVAL<\/span> HIGH_VALUE<br>--------------<span style=\"color: #008000\">-- -------- -------------------------------------<\/span><br>PART_01          <span style=\"color: #0000ff\">NO<\/span>       TO_DATE(<span style=\"color: #006080\">' 2016-05-01 00:00:00'<\/span>, ...<br>SYS_P17402       YES      TO_DATE(<span style=\"color: #006080\">' 2016-11-01 00:00:00'<\/span>, ...<br>SYS_P17403       YES      TO_DATE(<span style=\"color: #006080\">' 2016-12-01 00:00:00'<\/span>, ...<br>SYS_P17404       YES      TO_DATE(<span style=\"color: #006080\">' 2017-01-01 00:00:00'<\/span>, ...<\/pre>\n<p><\/div>\n<p>&#8220;PART_01&#8221; won&#8217;t have any rows in it, but it defines the interval for future partitions.  Now imagine some time goes by, and we don&#8217;t want the oldest month of data&#8230; so, let&#8217;s drop October, &#8216;SYS_P17402&#8217;:<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">SQL<\/span>&gt; <span style=\"color: #0000ff\">alter<\/span> <span style=\"color: #0000ff\">table<\/span> Log_Data_P <span style=\"color: #0000ff\">drop<\/span> partition SYS_P17402;<br> <br><span style=\"color: #0000ff\">Table<\/span> altered<br><br><span style=\"color: #0000ff\">select<\/span> PARTITION_NAME, <span style=\"color: #0000ff\">INTERVAL<\/span>, HIGH_VALUE<br>   <span style=\"color: #0000ff\">from<\/span> DBA_Tab_Partitions<br>   <span style=\"color: #0000ff\">where<\/span> TABLE_NAME = <span style=\"color: #006080\">'LOG_DATA_P'<\/span><br>   <span style=\"color: #0000ff\">order<\/span> <span style=\"color: #0000ff\">by<\/span> PARTITION_POSITION;<br> <br>PARTITION_NAME   <span style=\"color: #0000ff\">INTERVAL<\/span> HIGH_VALUE<br>--------------<span style=\"color: #008000\">-- -------- ------------------------------------<\/span><br>PART_01          <span style=\"color: #0000ff\">NO<\/span>       TO_DATE(<span style=\"color: #006080\">' 2016-05-01 00:00:00'<\/span>, ...<br>SYS_P17403       YES      TO_DATE(<span style=\"color: #006080\">' 2016-12-01 00:00:00'<\/span>, ...<br>SYS_P17404       YES      TO_DATE(<span style=\"color: #006080\">' 2017-01-01 00:00:00'<\/span>, ...<\/pre>\n<p><\/div>\n<p>What&#8217;s the earliest row in the table?<\/p>\n<div id=\"codeSnippetWrapper\">\n<pre id=\"codeSnippet\" style=\"border-top-style: none; font-size: 8pt; overflow: visible; border-left-style: none; font-family: 'Courier New', courier, monospace; width: 100%; border-bottom-style: none; color: black; padding-bottom: 0px; direction: ltr; text-align: left; padding-top: 0px; border-right-style: none; padding-left: 0px; margin: 0em; line-height: 12pt; padding-right: 0px; background-color: #f4f4f4\"><span style=\"color: #0000ff\">SQL<\/span>&gt; <span style=\"color: #0000ff\">select<\/span> <span style=\"color: #0000ff\">count<\/span>(1), <span style=\"color: #0000ff\">min<\/span>(LOG_TIME) <span style=\"color: #0000ff\">from<\/span> Log_Data_P;<br> <br>  <span style=\"color: #0000ff\">COUNT<\/span>(1)  <span style=\"color: #0000ff\">MIN<\/span>(LOG_TIME)<br>--------<span style=\"color: #008000\">--  -----------------------------------<\/span><br>    859906  01-NOV-16 12.00.02.397712 AM -05:00<\/pre>\n<p><\/div>\n<p>Success!<\/p>\n<p>Now we just need a way to calculate the name of the partition based on, say, SYSDATE-180 (or whatever our aging specification is) and we&#8217;re golden.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Consider a table, LOG_DATA, with lots of rows: select trunc( LOG_TIME, &#8216;MM&#8217; ) as MONTH, count(1) as Countfrom Log_Datagroup by trunc(LOG_TIME,&#8217;MM&#8217;)order by trunc(LOG_TIME,&#8217;MM&#8217;); Perhaps this would be better served by a table with RANGE partitioning, using the INTERVAL syntax of 11g to automatically create partitions as data accumulates? Unfortunately, LOG_TIME is data type timestamp with [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"class_list":["post-33","post","type-post","status-publish","format-standard","hentry","category-oracle","post-preview"],"_links":{"self":[{"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/posts\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/comments?post=33"}],"version-history":[{"count":0,"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/posts\/33\/revisions"}],"wp:attachment":[{"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/media?parent=33"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/categories?post=33"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/spacefold.com\/colin\/morethanfour\/wp-json\/wp\/v2\/tags?post=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}