Provide Delta files between versions of the Password DB for offline mirror updates
for offline mirrors it is important to be able to stay up to date - dropping the index and whole db only to reimport all 550M entries is a long time - for local offline copies the # of breaches may not be important but the new hashes are - can you provide delta files of the newly added SHA1s only for easier updates?
Love using the service btw!
would also keep your bandwith lower for people only needing the new stuff by downloading smaller files
Anonymous
shared this idea
Deltas would still be extremely large due to the prevalence counts on so many of the passwords changing. Best bet is to either download the new version or if that becomes inconvenient, hit the k-anonymity API.
-
Anonymous
commented
unless you link to what breach and want to show stats - the prevelance count is irrelevant but you just need the index of hashes. Atm we always have to download, drop the 2nd column, import - as there are no deltas we have to write a dedupe script or drop, reimport, reindex - it would be way more convenient for non API users to have a delta list of new uniquely added records
-
Anonymous
commented
Actually for some people the prevalence counts do not matter! We always strip them when loading because they are not really meaningful if you need to check for bad passwords only