Remix.run Logo
Prefer Strict Tables in SQLite(evanhahn.com)
85 points by ingve 2 hours ago | 31 comments
jll29 an hour ago | parent | next [-]

I'd like to see STRICT as the default.

That's pretty much the only disagreement with the SQLite developer, who is an amazing guy that wrote an amazing tool!

simonw 37 minutes ago | parent | next [-]

SQLite very rarely changes defaults because of their commitment to backwards compatibility. They don't want software written against SQLite 3.53 to suddenly start throwing errors when upgraded to 3.54 because suddenly `CREATE TABLE` is creating strict tables and the rest of the software breaks as a result.

ezst 2 minutes ago | parent [-]

Which seems reasonable. And those who care deeply will have no problem configuring it the specific way they want on their own project. Win-win.

ahartmetz 14 minutes ago | parent | prev | next [-]

There are more similar issues, like disabling foreign key constraints by default "for compatibility reasons". Makes me wonder if there was a time when SQLite supported foreign key syntax, but didn't actually implement the functionality.

formerly_proven 2 minutes ago | parent [-]

> This document describes the support for SQL foreign key constraints introduced in SQLite version 3.6.19 (2009-10-14).

mort96 43 minutes ago | parent | prev [-]

Yeah it's a really weird design decision. Why would I want the database to let me accidentally insert the wrong type? SQLite is mostly great but its philosophy towards type safety leaves something to be desired. I once had to clean up in a project where someone had accidentally stored the strings '1' and '0' in a Boolean column in code deployed to thousands of devices; not fun.

Another thing I dislike is the lack of timestamp types. Instead, you're expected to just use a text column and store a textual timestamp. Even worse, instead of using ISO, the standard date time functions produce strings on the form "yyyy-mm-dd HH:MM:SS" which you're just supposed to assume are in UTC. Why not at least give us "yyyy-mm-ddTHH:MM:SSZ"? Or, you know, a proper space efficient timestamp data type.

A truly great project, with some truly baffling design decisions.

3eb7988a1663 17 minutes ago | parent | next [-]

'0' and '1', while not ideal, seems fine to maintain? There is not even a true SQLite boolean type.

Then again, I have been subjected to Oracle nonsense for too long and have had to accept all of the boolean alternatives: 0,1,'0','1',Y,N,y,n,YES,NO,T,F, etc

masklinn 21 minutes ago | parent | prev | next [-]

> Instead, you're expected to just use a text column and store a textual timestamp.

You can actually use an integer column and store Unix timestamps (or floats for subsecond accuracy).

But yes, sqlite has very little types support and its default behaviour is very much unityped / dynamically typed which I also dislike. Same with having to enable foreign keys every time you open a connection.

formerly_proven 19 minutes ago | parent | prev [-]

If every time the SQLite team added a better default for something they changed it, we'd be at SQLite 11 by now and every application would contain at least six incompatible versions of SQLite.

petilon 37 minutes ago | parent | prev | next [-]

The downside of strict tables is that some data types are not available, such as Date.

Strict should really be the default. If a database is shared by multiple applications then you should be able to rely on the declared data type. If one application stores a string into a numeric column that breaks everyone else.

On the other hand, the main use case for SQLite is embedded databases. And that means only one application is using the database. In that scenario being able to evolve the schema (as opposed to creating a new database and copying the data over) can be seen as an advantage. The application's code knows what to expect in each column--including mixed data types.

e2le 18 minutes ago | parent | next [-]

> The downside of strict tables is that some data types are not available, such as Date.

There are only 5 datatypes in sqlite. INTEGER, TEXT, BLOB, REAL, and NUMERIC.

https://sqlite.org/datatype3.html

masklinn 6 minutes ago | parent [-]

numeric is not a type it’s an affinity, the underlying types are real and integer. That is why numeric is not valid on strict tables.

Ciantic 31 minutes ago | parent | prev | next [-]

SQLite has no date data type. Also SQLite has no way to call EXPLAIN on query and get the dummy type name either for arbitrary SELECT query, so you can't even infer it, if you were to use the dummy type name "DATE" or "DATETIME".

masklinn 18 minutes ago | parent | prev [-]

> some data types are not available, such as Date.

That’s not a type, you just get a numeric-affinity column.

cdmckay an hour ago | parent | prev | next [-]

I would’ve thought this was the default.

blixt 37 minutes ago | parent | prev | next [-]

I think I can see how dynamic data types make sense (eg flat key/value store), but my question would be:

What is least surprising? That INTEGER implicity accepts 'hello world' without error, or that you can't insert such a value unless you use a keyword like NONSTRICT or a type like ANY?

I would wager the vast majority of SQLite users if asked would probably not expect it to work.

tehlike an hour ago | parent | prev | next [-]

It really should be default, but it isn't due to backward compatibility (i assume).

poidos an hour ago | parent | next [-]

It’s a stated [0] goal of the project:

> SQLite strives to be flexible regarding the datatype of the content that it stores.

[0]: https://sqlite.org/stricttables.html

masklinn 17 minutes ago | parent [-]

You can be flexible with strict tables, type every column as ANY and you pretty much get back the original behaviour.

poidos 9 minutes ago | parent [-]

Sure, but you lose the representation of the developer’s intention that way. I would be pretty pissed off if I inherited a project and the schema was all ANYs.

drdexebtjl 2 minutes ago | parent | next [-]

“I intended this to be an integer but it could really be anything” is not very useful.

masklinn 2 minutes ago | parent | prev [-]

The intent of ANY is obviously that the values be flexible. That’s why it’s there if you need it.

tuvix an hour ago | parent | prev [-]

I’m kind of curious why the decision to have implicit casting like this was made in the first place. I can’t think of a single upside other than not having to type out cast(foo as bar)

rogerbinns 39 minutes ago | parent | next [-]

SQLite was originally started as a local database library for use during development for times when the main networked database was not available. It used dbm as the underlying storage mechanism, with the dbm API roughly being string keys with string values. ie all underlying values were actually stored as strings. The SQLite code would automatically do conversions - eg the plus operator would convert the strings to int or float, add them, and generate a stringified number as a result. The vast majority of implementation code did not have to care about types, and very local decisions could be made such as in the addition example.

TCL was used as a dev wrapper language at the time, and it functioned the same way.

It was only in mid-2004 that SQLite 3 was released which used its own storage backend, and that allowed for the 5 supported storage types (int64, string, bytes, float, null). It was API compatible (with minor adjustments) with the earlier SQLite 2, so the lack of static typing continued, otherwise everyone would have to rewrite their code. You do get dynamic typing, which hasn't been a problem for the vast majority of SQLite users.

Do remember that SQLite is competition for fopen, not Oracle / Postgres etc. It is trying to make things as effective as possible in that scenario. If you don't want numbers in your string column, then don't do that!

mb7733 8 minutes ago | parent | prev | next [-]

It's even worse than implicit casting, if the value can't be cast to the the column's type, it's just inserted without casting. Eg. into an integer column, '10' -> 10 and '1O' -> '1O' (hint: latter is uppercase o)

rogerbinns a minute ago | parent [-]

That is documented behaviour - think of it as making a best effort, and not losing the value.

As of January 2006 you could add CHECK constraints using the TYPEOF function to reject that at the SQL level. And it is your own code - there is no server - doing the insertions. As was common back then, protecting you from your own bugs was not a high priority for APIs!

notRobot an hour ago | parent | prev | next [-]

SQLite docs: The Advantages Of Flexible Typing: https://sqlite.org/flextypegood.html

pstuart an hour ago | parent | prev [-]

IIRC, the project started out as TCL code and it carried that vibe through to what it is today.

moron4hire 39 minutes ago | parent | prev | next [-]

Using Entity Framework, this doesn't come up as a particular issue, but I still wish it were strict by default because I expect there could be some performance optimizations made for de/serialization.

Eswo an hour ago | parent | prev | next [-]

really interesting, thanks

itsthecourier an hour ago | parent | prev [-]

about the use of ANY, that's perfect for tracking changes on an audit table per field