Ticket #1592 (closed defect: fixed)

Opened 22 months ago

Last modified 11 months ago

Oracle DataDict NUMBER and VARCHAR2 issues

Reported by: adrive Owned by: jwage
Priority: major Milestone: 1.2.0
Component: Schema Files Version: 1.0.3
Severity: abc Keywords:
Cc: Has Test: no
Status: Commit Approved Has Patch: no

Description (last modified by jwage) (diff)

Hello. I decide to create this ticket, because I am not sure, if the way I decide to solve some issues is right and I would like to discuss about it.

VARCHAR2 issues:

Oracle supports VARCHAR2 columns to be 4000 bytes long. So we can store 4000 CHARS in VARCHAR2 columns, but not unicode. So I decide to make a patch to allow store max. 2000 CHARS in VARCHAR2 columns to be able to store also unicode chars without getting an 4000 bytes exceed error.

Maybe the other solution woul be to specify unit parameter of column with two possible values: chars, bytes. And the user should make decission which one will be used. I can create a patch also for this possibility. But now, VARCHAR2 length is unlimited, which trigger errors when creating tables in database.

NUMBER issues:

Now when descibing number columns in MySQL or Posgresql should be specified:

bigint:
  type: integer(8)
integer:
  type: integer(4)
mediumint:
  type: integer(3)
smallint:
  type: integer(2)
tinyint:
  type: integer(1)
...

In MySQL or Pgsql this will generate integer(4) => INTEGER, integer(3) => MEDIUMINT, integer(2) => SMALLINT, integer(1) => TINYINT. As lenght means number of bytes the number will be stored in. Oracle doesn't support any of the *INT, even there are some synonyms for NUMBER (INTEGER, SMALLINT).

In oracle NUMBER datatype should have two params precisions and scale.

In Oracle integer(4) will be NUMBER(4) which means, that the biggest possible value for this column is 9999. As there are alredy some plugins (sfDoctrineGuard) that uses for id definition the lenght for integer(4) it means, that in Oracle you cannot have more then 9999 sfGuardUsers in your database.

Therefor I am suggesting to translating the size attribute of number columns to equivalents of BIGINT, MEDIUMINT... For example:

integer(1) to NUMBER(3) // max value 255 in unisgned TINYINT in MySQL
integer(2) to NUMBER(5) // max value 65535 in unsigned SMALLINT
integer(3) to NUMBER(8) // MEDIUMINT 16777215
integer(4) to NUMBER(10) // INT 2147483647
integer(8) to NUMBER(20) // BIGINT 18446744073709551615

Any size of integer definition larger than 8 and smaller than 38 will be translated tu NUMBER(n)

Of course, in datadict should be new type - number, and anyone who will use only Oracle should be able to define NUMBER(precision, scale) as he wants.

I will provide Oracle for Doctrine developers in a few days. But you will need Oracle client and compiled oci8 drivers.

Attachments

OracleDataDict.patch Download (10.1 KB) - added by adrive 22 months ago.
Oracle patch with VARCHAR2 max length 2000 CHARS
OracleNumberIssues.patch Download (12.8 KB) - added by adrive 22 months ago.
My patch for 1.1 with some new feature added (Import will recognize priamry keys columns)
MaybeMyFinal.patch Download (13.1 KB) - added by adrive 21 months ago.
Patch waiting for commit approval

Change History

Changed 22 months ago by adrive

Oracle patch with VARCHAR2 max length 2000 CHARS

Changed 22 months ago by jwage

  • milestone changed from 1.0.4 to 1.1.0

Changed 22 months ago by adrive

My patch for 1.1 with some new feature added (Import will recognize priamry keys columns)

Changed 22 months ago by adrive

I thought about the VARCHAR2 issue and would be the best way to let the user choose which unit will be used by default (bytes/chars) for VARCHAR2 columns. There should be an attribute for it.

Changed 21 months ago by adrive

So I create probably the last patch on this topic. It adds new way of hadling integers in Oracle DataDict described above. It also adds new feature - primary key recognition.

If the user will want to choose the unit of length at CHAR|VARCHAR2 while exporting with $conn->setParam('char_unit', 'BYTE|CHAR'); and VARCHAR2 definition will be exported with the unit VARCHAR2(10 CHAR) instead of the default VARCHAR2(10)`.

Please, review my latest patch and set status Commit approved on this ticket ;-)

If this patch will be approved, it will be good to know the place, where can we document at least new length of integers in oracle.

Changed 21 months ago by adrive

Patch waiting for commit approval

Changed 20 months ago by jwage

  • mystatus changed from Pending Core Response to Commit Approved
  • description modified (diff)
  • milestone changed from 1.1.0 to 1.0.5

Changed 20 months ago by jwage

Commit this to both 1.0 and 1.1. Be sure that all tests pass and double check your code to make sure it is not introducing new bugs.

Changed 20 months ago by jwage

  • milestone changed from 1.0.5 to 1.0.6

I have moved this to 1.0.6. We need confirmation that applying the patches doesn't break anything. If all is well then we can include this in 1.0.6 and 1.1 branch.

Changed 20 months ago by jwage

  • milestone changed from 1.0.6 to 1.0.7

Changed 18 months ago by jwage

  • milestone changed from 1.0.7 to 1.0.8

Changed 18 months ago by adrive

Sorry, that I haven't responded earlier.

I am sure, that this can't break anything.

This patch makes the oracle integers be "bigger" not smaller. So If someone have defined integer(1) and want it be NUMBER(1), after applying this patch it will be NUMBER(3). But I think, that this change is quite significant, so it should be moved only to 1.1 branch and I think it need some lines in what's new in 1.1 or documentation.

The main idea of this patch is to be compatible with other databases. I think, that many devs using doctrine are also using symfony and it's plugins that are mainly tested on MySQL. So there is no chance to have for example more than 9999 sfGuardUsers in Oracle now (yes, the column should be extended in db manually, but this is not the reason why we want to use doctrine, to work manually).

If somebody rely on the number size and needs eg. NUMBER(1) it should be defined as type: decimal, length: 1, scale: 0 and it generates NUMBER(1,0). I know that this isn't the best practice.

It will be quite good to know the opinion of other devs using oracle on this topic, but I wrote to few of them that are postig oracle specific bugs, but I didn't get any response.

Maybe there should be another way: a compat attribute for Oracle driver that will cause this behavior, or something else. But I am not familiar with the attributes in doctrine core.

Changed 18 months ago by jwage

  • milestone changed from 1.0.8 to 1.2.0

Changed 13 months ago by jwage

  • severity set to abc

We can include this in 1.2 but now the patch won't apply. Can you adjust and reattach for the 1.2 branch. Also, delete all the old patches so nobody is confused as to what patch is the latest. You can commit these changes to 1.2 yourself.

Changed 13 months ago by adrive

  • status changed from new to closed
  • resolution set to fixed

Fixed in r5938

Changed 11 months ago by kwutzke

I'm pretty sure that the mappings from Doctrine types to Oracle are true for UNSIGNED types only. For every unsigned max value starting with a 1, halving the max value will yield one precision less, because that suffices.

From the table

integer(1) to NUMBER(3) // max value 255 in unisgned TINYINT in MySQL
integer(2) to NUMBER(5) // max value 65535 in unsigned SMALLINT
integer(3) to NUMBER(8) // MEDIUMINT 16777215
integer(4) to NUMBER(10) // INT 2147483647
integer(8) to NUMBER(20) // BIGINT 18446744073709551615

this affects integer(3) and integer(8). So, for any SIGNED types, which should be the default (because standard SQL doesn't even know unsigned types), the precision should be:

integer(1) to NUMBER(3)  //127
integer(2) to NUMBER(5)  //32767
integer(3) to NUMBER(7)  //8388607
integer(4) to NUMBER(10) //2147483647
integer(8) to NUMBER(19) //9223372036854775807

Don't know how much this affects the solution. I didn't want reopen it.

Note: See TracTickets for help on using tickets.