Here’s a problem that appeared recently on the Orace Developer forum showing one of the classic symptons of new features namely that “mix and match” often runs into problems. This example has been a long time coming so “new” is something of a misnomer, but the alternative source of unlucky collisions is in the “rare” features – in this case Spatial. (It’s possible that the problem is not restricted to just Spatial but it probably does require a somewhat exotic data type.)
The problem appeared in a thread on the Oracle Developer Forum from someone who was trying to drop a pair of columns and finding that the statement failed with a surprising choice of error: ORA-00904: Invalid Identifier. The surprising thing about this error was that the named identifier was clearly not invalid. The suspicion that that this was an example of “new features colliding” was that the columns to be dropped were virtual columns based on a real column of the table that had been declared as an object type defined in the MDSYS (Spatial) schema.
Conveniently the author had supplied a short, simple, script to demonstrate the issue, so I copied it and modified it a bit to do a few tests around the edges of the problem. Here’s the code that I used to start my investigation:
rem
rem Script: drop_col_bug.sql
rem Author: Jonathan Lewis/User_77G7L
rem Dated: Mar 2022
rem Purpose:
rem
rem Last tested
rem 21.3.0.0 Still broken
rem 19.11.0.0
rem
create table xxx (
v1 varchar2(10),
n1 number,
shape mdsys.sdo_geometry,
x_value number generated always as (mdsys.sdo_geom.sdo_pointonsurface(shape,0.005).sdo_point.x) virtual,
y_value number generated always as (mdsys.sdo_geom.sdo_pointonsurface(shape,0.005).sdo_point.y) virtual,
v2 varchar2(10),
n2 number,
n3 number
)
segment creation immediate
;
insert into xxx(v1, n1, v2, n2, n3) values('z',1,'y',2,3);
update xxx set
shape = sdo_geometry(
2003, -- two-dimensional polygon
null,
null,
sdo_elem_info_array(1,1003,3), -- one rectangle (1003 = exterior)
sdo_ordinate_array(1,1, 5,7) -- only 2 points needed to define rectangle
)
;
commit;
alter table xxx drop (x_value, y_value) ;
The modifications I made from the original code are:
- I’ve removed a couple of redundant sets of parentheses from the virtual column definitions
- I’ve added a few columns before, in between, and after the virtual columns
- I’ve used “segment creation immediate”
- I’ve inserted a row into the table
The last two are simply to ensure that I have data segments and at least one item for the drop to work on – just in case it’s a run-time problem being reported as a parse time issue.
The extra columns are to test whether the type and position of the column I drop affects the outcome, and the change in parentheses is purely aesthetic.
Here’s the result of the attempt to drop the virtual columns:
alter table xxx drop (x_value, y_value)
*
ERROR at line 1:
ORA-00904: "MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE": invalid identifier
This is a little strange since I have used the packaged function mdsys.sdo_geom.sdo_pointonsurface() to define the virtual columns and Oracle didn’t complain when I created the column. (Part of the reason I had reduced the original parentheses was to check that the compiler hadn’t got confused by an excess of paretheses).
As a quick “what if” test I tried using the alternative syntax for drop column that you can use with just one column:
SQL> alter table xxx drop column x_value;
alter table xxx drop column x_value
*
ERROR at line 1:
ORA-00904: "MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE": invalid identifier
What about trying to set the column unused before dropping all unused columns?
SQL> alter table xxx set unused column x_value;
alter table xxx set unused column x_value
*
ERROR at line 1:
ORA-00904: "MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE": invalid identifier
So is the problem restricted to the virtual columns – what happens if I try to drop a column from the end of the table, what about the one between the two virtual columns, how about a column that appears before even the shape column? Nothing changes:
SQL> alter table xxx drop column v1;
alter table xxx drop column v1
*
ERROR at line 1:
ORA-00904: "MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE": invalid identifier
SQL> alter table xxx set unused column v1;
alter table xxx set unused column v1
*
ERROR at line 1:
ORA-00904: "MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE": invalid identifier
What if I have only one of the virtual columns? No difference.
What if I don’t have either of the virtual columns? Finally I can drop any column I like from the table (including the shape column). Not that that’s much use to the user.
You can’t set unused or drop any columns in the table thanks to an error that looks as if it’s associated with the definition of those virtual columns.
Workaround
Is there any way to bypass the problem and still store the information we need (until we want to drop it). Let’s start by taking a look at the way Oracle has used our table definition to create column definitions, just in case that gives us a clue:
select
column_id id, segment_column_id seg_id, internal_column_id int_id,
column_name, data_type, data_default
from
user_tab_cols
where
table_name = 'XXX'
order by
column_id,
internal_column_id
/
ID SEG_ID INT_ID COLUMN_NAME DATA_TYPE DATA_DEFAULT
---------- ---------- ---------- -------------------- ------------------------- --------------------------------------------------------------------------------
1 1 1 V1 VARCHAR2
2 2 2 N1 NUMBER
3 3 3 SHAPE SDO_GEOMETRY
3 4 4 SYS_NC00004$ NUMBER
3 5 5 SYS_NC00005$ NUMBER
3 6 6 SYS_NC00006$ NUMBER
3 7 7 SYS_NC00007$ NUMBER
3 8 8 SYS_NC00008$ NUMBER
3 9 9 SYS_NC00009$ SDO_ELEM_INFO_ARRAY
3 10 10 SYS_NC00010$ SDO_ORDINATE_ARRAY
4 11 X_VALUE NUMBER (("MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE"("SHAPE",0.005))."SDO_POINT")."X"
5 12 Y_VALUE NUMBER (("MDSYS"."SDO_GEOM"."SDO_POINTONSURFACE"("SHAPE",0.005))."SDO_POINT")."Y"
6 11 13 V2 VARCHAR2
7 12 14 N2 NUMBER
8 13 15 N3 NUMBER
15 rows selected.
There’s quite a lot going on there in terms of columns hidden behind the sdo_geometry type. In fact internal columns 9 and 10 might prompt you to look for other objects like table types or LOBs:
SQL> select column_name, segment_name, index_name from user_lobs where table_name = 'XXX';
COLUMN_NAME SEGMENT_NAME INDEX_NAME
------------------------------ ------------------------------ ------------------------------
"SHAPE"."SDO_ELEM_INFO" SYS_LOB0000100168C00009$$ SYS_IL0000100168C00009$$
"SHAPE"."SDO_ORDINATES" SYS_LOB0000100168C00010$$ SYS_IL0000100168C00010$$
2 rows selected.
But the interesting detail is the data_default column for our two virtual columns – which have more parentheses than the original definitions. Perhaps the storage of the expression has gone wrong (as happened in an older version of Oracle with case expressions) and is causing the ORA-00904 error to appear. So let’s try selecting data from the table using the expression stored in data dictionary:
select
((MDSYS.SDO_GEOM.SDO_POINTONSURFACE(SHAPE,0.005)).SDO_POINT).X old_x,
mdsys.sdo_geom.sdo_pointonsurface(shape,0.005).sdo_point.x new_x,
((MDSYS.SDO_GEOM.SDO_POINTONSURFACE(SHAPE,0.005)).SDO_POINT).Y old_y
from
xxx
/
OLD_X NEW_X OLD_Y
---------- ---------- ----------
1 1 1
1 row selected.
No syntax error there – as far as a simple select is concerned. I’ve included my tidier format for the x_value column aligned with the resulting stored value (with all the double quotes removed – though I’ve also tested it with the quotes in place) – and the only significant visual difference is the number of parentheses, so maybe that’s a clue. In particular we note that the error reports “MDSYS”.”SDO_GEOM”.”SDO_POINTONSURFACE” as the invalid identifier and the first time an extra (close) parenthesis appears is just after that function call. Maybe (for no good reason) the code path involved with handling column data during a drop/set unused call is getting confused by parentheses. So let’s try to reduce the complexity of the expression by hiding it inside a local function.
First attempt – create a function to return an sdo_point_type and define the virtual columns to expose the X and Y values from the point:
create or replace function my_point(
inshape in mdsys.sdo_geometry,
intolerance in number
)
return mdsys.sdo_point_type
deterministic
as
begin
return mdsys.sdo_geom.sdo_pointonsurface(inshape, intolerance).sdo_point;
end;
/
x_value number generated always as (my_point(shape,0.005).x) virtual,
y_value number generated always as (my_point(shape,0.005).y) virtual,
This approach still produces an ORA-00904, though the invalid identifier becomes “TEST_USER”.”MY_POINT”.
Second attempt – two separate functions, one for the x value, one for the y value:
create or replace function my_x(
inshape in mdsys.sdo_geometry,
intolerance in number
)
return number
deterministic
as
begin
return mdsys.sdo_geom.sdo_pointonsurface(inshape, intolerance).sdo_point.x;
end;
/
show errors
create or replace function my_y(
inshape in mdsys.sdo_geometry,
intolerance in number
)
return number
deterministic
as
begin
return mdsys.sdo_geom.sdo_pointonsurface(inshape, intolerance).sdo_point.y;
end;
/
x_value number generated always as (my_x(shape, 0.005)) virtual,
y_value number generated always as (my_y(shape, 0.005)) virtual,
This worked so, finally, I looked at the SQL Language reference manual to see if there were any restrictions on virtual columns that might explain the problem I had had with all the previous definitions (and, yes, I know I should have done that right after the first failure) and I found the following:
- The virtual column cannot be an Oracle supplied data type, a user-defined type, or LOB or
LONGRAW.
None of my virtual column definitions returned an Oracle supplied data type or a user-defined data type. But would the restriction also apply to single attributes of such a data type, or has the complexity of spatial data types found a hole in the code? And the answer to that question is a whole new blog note waiting to be finish (because I’d forgotten what a pain it was to use object types in simple SQL.)
