github-starred/acebase

Fork 0

mirror of https://github.com/appy-one/acebase.git synced 2026-05-25 06:02:14 -06:00

[GH-ISSUE #21] IndexedDb extra rows #20

New issue

Closed

opened 2026-05-23 08:25:44 -06:00 by gitea-mirror · 7 comments

gitea-mirror commented

2026-05-23 08:25:44 -06:00

Owner

Originally created by @clibu on GitHub (Mar 16, 2021).
Original GitHub issue: https://github.com/appy-one/acebase/issues/21

I have ab object such as:

user: {
        emailAddr: "fred@123.com"
        passwordHash: "$2a$10$/adc6zcERXdkTnSXNEQ6suac1S48tK3PsfPqgszBN5PUTN1hnG2Ia"
        state: "signedIn"
        token: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI2aXhvUkpqdHhsVjk1QUZkYVFiUlAiLCJwYXNzd29yZEhhc2giOiIkMmEkMTAkL2FkYzZ6Y0VSWGR"
        userId: "6ixoRJ"
        userName: "blah"
}

When I do db.ref( path ).update( user ) IndexedDb content table has three rows:

"clibu_users/6ixoRJjtxlV95AFdaQbRP": {userId: "6ixoRJjtxlV95AFdaQbRP", userName: "associated_sapphire_hippopotamus", emailAddr: "fred@123.com", state: "signedIn", id: "6ixoRJjtxlV95AFdaQbRP"}
"clibu_users/6ixoRJjtxlV95AFdaQbRP/passwordHash": "$2a$10$/adc6zcERXdkTnSXNEQ6suac1S48tK3PsfPqgszBN5PUTN1hnG2Ia"
"clibu_users/6ixoRJjtxlV95AFdaQbRP/token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI2aXhvU

passwordHash and token are stored in extra rows using there key is in the path and just their value in the Value clm.

I'm curious as to why it works this way and am concerned about the extra rows wasting space.

Along similar lines the nodes also has these extra rows.

Originally created by @clibu on GitHub (Mar 16, 2021). Original GitHub issue: https://github.com/appy-one/acebase/issues/21 I have ab object such as: ```` user: { emailAddr: "fred@123.com" passwordHash: "$2a$10$/adc6zcERXdkTnSXNEQ6suac1S48tK3PsfPqgszBN5PUTN1hnG2Ia" state: "signedIn" token: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI2aXhvUkpqdHhsVjk1QUZkYVFiUlAiLCJwYXNzd29yZEhhc2giOiIkMmEkMTAkL2FkYzZ6Y0VSWGR" userId: "6ixoRJ" userName: "blah" } ```` When I do ```db.ref( path ).update( user )``` IndexedDb ``content`` table has three rows: ```` "clibu_users/6ixoRJjtxlV95AFdaQbRP": {userId: "6ixoRJjtxlV95AFdaQbRP", userName: "associated_sapphire_hippopotamus", emailAddr: "fred@123.com", state: "signedIn", id: "6ixoRJjtxlV95AFdaQbRP"} "clibu_users/6ixoRJjtxlV95AFdaQbRP/passwordHash": "$2a$10$/adc6zcERXdkTnSXNEQ6suac1S48tK3PsfPqgszBN5PUTN1hnG2Ia" "clibu_users/6ixoRJjtxlV95AFdaQbRP/token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI2aXhvU ```` ``passwordHash`` and ``token`` are stored in extra rows using there key is in the path and just their value in the Value clm. I'm curious as to why it works this way and am concerned about the extra rows wasting space. Along similar lines the ``nodes`` also has these extra rows.

gitea-mirror closed this issue

2026-05-23 08:25:45 -06:00

gitea-mirror commented

2026-05-23 08:25:48 -06:00

Author

Owner

@appy-one commented on GitHub (Mar 16, 2021):

This is done to keep the individual records small and fast to read/write. Large strings (>50 chars) are stored in their own records for this reason, just like child objects and arrays are stored in their own records

@appy-one commented on GitHub (Mar 16, 2021): This is done to keep the individual records small and fast to read/write. Large strings (>50 chars) are stored in their own records for this reason, just like child objects and arrays are stored in their own records

gitea-mirror commented

2026-05-23 08:25:49 -06:00

Author

Owner

@clibu commented on GitHub (Mar 16, 2021):

Ok, understood. However this may not always be an advantage. For example if I want to read the user record then acebase needs to read 3 records and assemble them into one.

I'm guessing that the 50 char limit is fixed, if not could it be an option.

I'm concerned about IndexedDb size as Browsers have quota's and will discard db's at will and without warning.

@clibu commented on GitHub (Mar 16, 2021): Ok, understood. However this may not always be an advantage. For example if I want to read the user record then acebase needs to read 3 records and assemble them into one. I'm guessing that the 50 char limit is fixed, if not could it be an option. I'm concerned about IndexedDb size as Browsers have quota's and will discard db's at will and without warning.

gitea-mirror commented

2026-05-23 08:25:49 -06:00

Author

Owner

@appy-one commented on GitHub (Mar 16, 2021):

No need to worry about IndexedDB quota, there is not really such a thing. There is a total browser storage limit of 50% of the disk's free space. If the browser starts cleaning up, it will remove all stored data per site until requirements are met.

One thing to always keep in mind - storing data in the browser should only be done to speed up your app and allow offline usage, not to store critical info. Gone is gone. Always sync important data with a server db. Don't worry about a few extra bytes used by storing larger strings in their own records. Doing this does significantly speed up db reads and writes. Merging records into 1 is what AceBase always does - it also has to do this for child objects (also stored in their own records), their child objects, etc. If this would not be done, your entire data would have to be stored in 1 huge single record.

The max amount of chars is currently not configurable through AceBase.WithIndexedDB. When the AceBase constructor is used (with binary, sqlite, sql server, or custom storage), it is possible by setting maxInlineValueSize in the storage settings, eg: const db = new AceBase('mydb', { storage: { maxInlineValueSize: 100 } })

@appy-one commented on GitHub (Mar 16, 2021): No need to worry about IndexedDB quota, there is not really such a thing. There is a total browser storage limit of 50% of the disk's free space. If the browser starts cleaning up, it will remove all stored data per site until requirements are met. One thing to always keep in mind - storing data in the browser should only be done to speed up your app and allow offline usage, not to store critical info. Gone is gone. Always sync important data with a server db. Don't worry about a few extra bytes used by storing larger strings in their own records. Doing this does significantly speed up db reads and writes. Merging records into 1 is what AceBase always does - it also has to do this for child objects (also stored in their own records), their child objects, etc. If this would not be done, your entire data would have to be stored in 1 huge single record. The max amount of chars is currently not configurable through ```AceBase.WithIndexedDB```. When the ```AceBase``` constructor is used (with binary, sqlite, sql server, or custom storage), it is possible by setting ```maxInlineValueSize``` in the storage settings, eg: ```const db = new AceBase('mydb', { storage: { maxInlineValueSize: 100 } })```

gitea-mirror commented

2026-05-23 08:25:50 -06:00

Author

Owner

@clibu commented on GitHub (Mar 16, 2021):

The Knowledge Base / Note taking app I'm developing has been designed from the ground up to work completely offline. It also syncs to a master server database whenever it comes on line. That said I assume most users will be online most of the time.

If the Browser does evict the database(s) then a fresh copy is obtained from the server when the user next goes online. I can envisage DB's in the Gigabyte range for some users.

I'd welcome having the maxInlineValueSize option in IndexedDb. I could see this reducing disk space by 20+% On the server I don't see this as an issue. Also speed isn't an issue in my use case seeing the app runs locally, which is way faster than server round trips etc.

@clibu commented on GitHub (Mar 16, 2021): The Knowledge Base / Note taking app I'm developing has been designed from the ground up to work completely offline. It also syncs to a master server database whenever it comes on line. That said I assume most users will be online most of the time. If the Browser does evict the database(s) then a fresh copy is obtained from the server when the user next goes online. I can envisage DB's in the Gigabyte range for some users. I'd welcome having the ```maxInlineValueSize``` option in IndexedDb. I could see this reducing disk space by 20+% On the server I don't see this as an issue. Also speed isn't an issue in my use case seeing the app runs locally, which is way faster than server round trips etc.

gitea-mirror commented

2026-05-23 08:25:50 -06:00

Author

Owner

@appy-one commented on GitHub (Mar 17, 2021):

Ok. I don't mind adding the option to AceBase.WithIndexedDB, not much effort 🙂

@appy-one commented on GitHub (Mar 17, 2021): Ok. I don't mind adding the option to ```AceBase.WithIndexedDB```, not much effort 🙂

gitea-mirror commented

2026-05-23 08:25:51 -06:00

Author

Owner

@clibu commented on GitHub (Mar 17, 2021):

👍👍 Thanks.

@clibu commented on GitHub (Mar 17, 2021): 👍👍 Thanks.

gitea-mirror commented

2026-05-23 08:25:52 -06:00

Author

Owner

@appy-one commented on GitHub (Mar 18, 2021):

I've added and published this in v1.2.5
I still recommend using the default setting of 50, but to change it use:

const db = AceBase.WithIndexedDB('mydb', { maxInlineValueSize: 2500 });

@appy-one commented on GitHub (Mar 18, 2021): I've added and published this in v1.2.5 I still recommend using the default setting of 50, but to change it use: ```js const db = AceBase.WithIndexedDB('mydb', { maxInlineValueSize: 2500 }); ```