aboutsummaryrefslogtreecommitdiff
path: root/src/blockchain_db/lmdb (follow)
AgeCommit message (Collapse)AuthorFilesLines
2020-12-29rpc: limit the number of txes for get_blocks.binmoneromooo-monero2-4/+9
2020-08-17Avoid some temporary strings when reading off the databasemoneromooo-monero2-30/+25
2020-07-19Merge pull request #6565Alexander Blair1-1/+1
72cdfa4a2 fix a few typos in error messages (moneromooo-monero)
2020-07-19Merge pull request #6536Alexander Blair2-0/+30
bd9653663 db_lmdb: test for mmap support at init time (moneromooo-monero)
2020-07-19Merge pull request #6534Alexander Blair2-0/+40
7bd66b01b daemon: guard against rare 'difficulty drift' bug with checkpoints and recalculation (stoffu)
2020-06-09daemon: guard against rare 'difficulty drift' bug with checkpoints and ↵stoffu2-0/+40
recalculation On startup, it checks against the difficulty checkpoints, and if any mismatch is found, recalculates all the blocks with wrong difficulties. Additionally, once a week it recalculates difficulties of blocks after the last difficulty checkpoint.
2020-05-19fix a few typos in error messagesmoneromooo-monero1-1/+1
Reported by adrelanos
2020-05-16db_lmdb: test for mmap support at init timemoneromooo-monero2-0/+30
It'll make it clearer when a DB init failure is due to being on a filesystem which does not support mmap
2020-05-06Update copyright year to 2020SomaticFanatic2-2/+2
Update copyright year to 2020
2020-04-10Merge pull request #6358luigi11112-0/+99
8958b4e blockchain_db: faster fetching of consecutive txes (moneromooo-monero)
2020-04-04Merge pull request #6384luigi11111-0/+1
ec01077 db_lmdb: fix race crash using a stale cursor (moneromooo-monero)
2020-03-12db_lmdb: fix race crash using a stale cursormoneromooo-monero1-0/+1
If a db resize happened, the txpool meta cursor might be stale, and was not being renewed when necessary. It would cause this SEGSEGV: in mdb_cursor_set () in mdb_cursor_get () in cryptonote::BlockchainLMDB::get_txpool_tx_blob(crypto::hash const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, cryptonote::relay_category) const () in cryptonote::tx_memory_pool::get_transaction(crypto::hash const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, cryptonote::relay_category) const () in cryptonote::t_cryptonote_protocol_handler<cryptonote::core>::handle_notify_new_fluffy_block(int, epee::misc_utils::struct_init<cryptonote::NOTIFY_NEW_FLUFFY_BLOCK::request_t>&, cryptonote::cryptonote_connection_context&) ()
2020-02-27blockchain_db: faster fetching of consecutive txesmoneromooo-monero2-0/+99
Useful for wallet refresh or node sync
2020-02-17remove empty statementsInterchained1-2/+2
Cleaning up a little around the code base.
2020-01-25Merge pull request #6140Alexander Blair2-0/+43
08635a08 blockchain: speedup fetching pruned contiguous tx blobs (moneromooo-monero)
2020-01-24Merge pull request #6157Alexander Blair1-1/+9
402dd207 db_lmdb: guard against non NUL terminated keys (moneromooo-monero)
2019-11-19db_lmdb: guard against non NUL terminated keysmoneromooo-monero1-1/+9
2019-11-15blockchain: speedup fetching pruned contiguous tx blobsmoneromooo-monero2-0/+43
About twice as fast, very roughly
2019-11-02Adding support for hidden (anonymity) txpoolLee Clagett2-17/+41
2019-10-17db_lmdb: an empty prunable data record means the tx is not prunedmoneromooo-monero1-2/+0
as opposed to an absent record
2019-09-27monerod can now sync from pruned blocksmoneromooo-monero1-2/+4
If the peer (whether pruned or not itself) supports sending pruned blocks to syncing nodes, the pruned version will be sent along with the hash of the pruned data and the block weight. The original tx hashes can be reconstructed from the pruned txes and theur prunable data hash. Those hashes and the block weights are hashes and checked against the set of precompiled hashes, ensuring the data we received is the original data. It is currently not possible to use this system when not using the set of precompiled hashes, since block weights can not otherwise be checked for validity. This is off by default for now, and is enabled by --sync-pruned-blocks
2019-09-04db_lmdb: print percentages as percentages, not ratiosmoneromooo-monero1-1/+1
2019-08-18Increase max_dbs from 20 to 32Howard Chu1-1/+1
We've added a lot of new indices recently, and 20 isn't enough for them plus new DBs opened during format migrations.
2019-07-24Merge pull request #5524luigi11112-0/+168
06b8f29 blockchain: keep alternative blocks in LMDB (moneromooo-monero)
2019-07-24Merge pull request #5502luigi11111-1/+1
25a7cfd add a few checks where it seems appropriate (moneromooo-monero) 1a66a86 remove unused code (moneromooo-monero)
2019-06-15db_lmdb: commit pruning txn at checkpointsmoneromooo-monero1-1/+50
to avoid errors when the txn is too large
2019-06-14ensure no NULL is passed to memcpymoneromooo-monero1-2/+2
NULL is valid when size is 0, but memcpy uses nonnull attributes, so let's not poke the bear
2019-05-08blockchain: keep alternative blocks in LMDBmoneromooo-monero2-0/+168
Alternative blocks are cleared on startup unless --keep-alt-blocks is passed on the command line
2019-05-01fix wide difficulty conversion with some versions of boostmoneromooo-monero1-2/+2
2019-04-29add a few checks where it seems appropriatemoneromooo-monero1-1/+1
2019-04-14blockchain_db: fix db txn ending too earlymoneromooo-monero2-48/+39
The db txn in add_block ending caused the entire overarching batch txn to stop. Also add a new guard class so a db txn can be stopped in the face of exceptions. Also use a read only db txn in init when the db itself is read only, and do not save the max tx size in that case.
2019-04-05cryptonote: rework block blob size sanity checkmoneromooo-monero2-0/+55
Use the actual block weight limit, assuming that weight is always greater or equal to size
2019-03-24Make difficulty 128 bit instead of 64 bitmoneromooo-monero2-7/+155
Based on Boolberry work by: jahrsg <jahr@jahr.me> cr.zoidberg <crypto.zoidberg@gmail.com>
2019-03-21Merge pull request #5256Riccardo Spagni2-0/+70
4b21d38d blockchain: speed up getting N blocks weights/long term weights (moneromooo-monero)
2019-03-08blockchain: speed up getting N blocks weights/long term weightsmoneromooo-monero2-0/+70
2019-03-05Update 2019 copyrightbinaryFate2-2/+2
2019-03-05Merge pull request #5119Riccardo Spagni1-5/+5
f024a10b db_lmdb: make mdb_block_info format conversion more future proof (moneromooo-monero)
2019-03-05db_lmdb: make mdb_block_info format conversion more future proofmoneromooo-monero1-5/+5
If mdb_block_info changes again, the v2 to v3 conversion would convert to an incorrect format.
2019-03-05save some database calls when getting top block hash and heightmoneromooo-monero2-2/+4
2019-03-05Avoid repeated (de)serialization when syncingmoneromooo-monero2-9/+10
2019-03-05db: speedup block additionmoneromooo-monero1-11/+16
by avoiding repeated (de)serialization
2019-03-04Merge pull request #5081Riccardo Spagni1-3/+3
d78addcb db_lmdb: don't unnecessarily cast to double on the way to uint64_t (moneromooo-monero)
2019-03-04Merge pull request #5175Riccardo Spagni1-0/+1
85088d9f db_lmdb: fix missing mdb_dbi_close in migration (moneromooo-monero)
2019-03-04ArticMine's new block weight algorithmmoneromooo-monero2-6/+214
This curbs runaway growth while still allowing substantial spikes in block weight Original specification from ArticMine: here is the scaling proposal Define: LongTermBlockWeight Before fork: LongTermBlockWeight = BlockWeight At or after fork: LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight) Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time. Define: LongTermEffectiveMedianBlockWeight LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight)) Change Definition of EffectiveMedianBlockWeight From (current definition) EffectiveMedianBlockWeight = max(300000, MedianOverPrevious100Blocks(BlockWeight)) To (proposed definition) EffectiveMedianBlockWeight = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight) Notes: 1) There are no other changes to the existing penalty formula, median calculation, fees etc. 2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork. 3) When the EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty. Note: the long term block weight is stored in the database, but not in the actual block itself, since it requires recalculating anyway for verification.
2019-02-21db_lmdb: fix missing mdb_dbi_close in migrationmoneromooo-monero1-0/+1
Fixed by hyc
2019-01-22Pruningmoneromooo-monero2-23/+405
The blockchain prunes seven eighths of prunable tx data. This saves about two thirds of the blockchain size, while keeping the node useful as a sync source for an eighth of the blockchain. No other data is currently pruned. There are three ways to prune a blockchain: - run monerod with --prune-blockchain - run "prune_blockchain" in the monerod console - run the monero-blockchain-prune utility The first two will prune in place. Due to how LMDB works, this will not reduce the blockchain size on disk. Instead, it will mark parts of the file as free, so that future data will use that free space, causing the file to not grow until free space grows scarce. The third way will create a second database, a pruned copy of the original one. Since this is a new file, this one will be smaller than the original one. Once the database is pruned, it will stay pruned as it syncs. That is, there is no need to use --prune-blockchain again, etc.
2019-01-19db_lmdb: don't unnecessarily cast to double on the way to uint64_tmoneromooo-monero1-3/+3
2019-01-16blockchain_db: allow getting output keys without commitmentmoneromooo-monero2-3/+4
Since the commitment has to be calculated for non rct outputs, it slows down a lot unnecessarily if we don't need it
2019-01-16Merge pull request #4984Riccardo Spagni2-22/+28
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-07Make get_output_key method constmoneroexamples2-4/+4
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs. Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2018-12-31Merge pull request #4946luigi11112-50/+8
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero) ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero) 8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero) 5511563 db_lmdb: avoid pointless division (moneromooo-monero) d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero) 9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
2018-12-18blockchain_db: speedup tx output gatheringmoneromooo-monero2-22/+28
We know all the data we'll want for getblocks.bin is contiguous
2018-12-12Merge pull request #4901Riccardo Spagni2-4/+68
a48f2dab blockchain_prune_known_spent_data: blackball file is now optional (moneromooo-monero) 17b45725 Outputs where all amounts are known spent can now be pruned (moneromooo-monero)
2018-12-05db_lmdb: avoid pointless divisionmoneromooo-monero1-1/+1
2018-12-05db_lmdb: inline check_open, it's trivial and called everywheremoneromooo-monero2-8/+7
2018-12-05blockchain_db: remove a couple unused functionsmoneromooo-monero2-41/+0
2018-12-04Merge pull request #4872Riccardo Spagni1-10/+6
fc99f177 lmdb: fix gcc 7.3.0 'implicit-fallthrough' warning (xiphon)
2018-12-04Merge pull request #4854Riccardo Spagni1-2/+0
bd98e99c Removed a lot of unnecessary includes (Martijn Otto)
2018-11-27Outputs where all amounts are known spent can now be prunedmoneromooo-monero2-4/+68
Only for pre rct for obvious reasons. Note: DO NOT use a known spent list which includes outputs which are not known spent. If the list includes any output that's just strongly thought to be spent, but not provably so, you risk finding yourself unable to sync past the point where that output is spent. I estimate only 200 MB saved on current mainnet though, unless the new blackballing rule unearths a good amount of large-amount-set extra spent outs.
2018-11-26Merge pull request #4821Riccardo Spagni2-6/+10
fc98f7a0 rpc: speedup get_outs.bin (moneromooo-monero)
2018-11-26rpc: speedup get_outs.binmoneromooo-monero2-6/+10
2018-11-26Merge pull request #4830Riccardo Spagni1-0/+9
dffec258 db_lmdb: error out if the db needs migration in read only mode (moneromooo-monero)
2018-11-19lmdb: fix gcc 7.3.0 'implicit-fallthrough' warningxiphon1-10/+6
2018-11-15Removed a lot of unnecessary includesMartijn Otto1-2/+0
2018-11-14Merge pull request #4769Riccardo Spagni2-21/+26
5808530f blockchain: remove unused output_scan_worker parameter (moneromooo-monero) 1426209a blockchain: don't run threads if we have just one function to run (moneromooo-monero) 6f7a5fd4 db_lmdb: slight speedup getting array data from the blockchain (moneromooo-monero) 99fbe100 db_lmdb: save some string copies for readonly db keys/values (moneromooo-monero) bf31447e tx_pool: speed up take_tx for transactions from blocks (moneromooo-monero) 4f005a77 tx_pool: remove unnecessary get_transaction_hash (moneromooo-monero) 593ef598 perf_timer: call reserve on new timer array (moneromooo-monero) 6ecc99ad core: avoid unnecessary tx/blob conversions (moneromooo-monero) 00cc1a16 unit_tests: notify test special case for the usual weirdo (moneromooo-monero)
2018-11-09db_lmdb: error out if the db needs migration in read only modemoneromooo-monero1-0/+9
2018-11-04db_lmdb: slight speedup getting array data from the blockchainmoneromooo-monero1-6/+8
2018-11-02db_lmdb: log which output was not found in get_output_keymoneromooo-monero1-1/+2
2018-11-01Merge pull request #4664Riccardo Spagni1-9/+23
61304151 db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup (moneromooo-monero)
2018-11-01db_lmdb: save some string copies for readonly db keys/valuesmoneromooo-monero1-11/+16
2018-11-01core: avoid unnecessary tx/blob conversionsmoneromooo-monero2-5/+3
2018-10-19db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedupmoneromooo-monero1-9/+23
2018-10-17Update db_lmdb.cppCactii11-1/+1
2018-10-02Fix rtxn usage in BlockchainLMDB::get_estimated_batch_sizeHoward Chu1-2/+2
Should only stop the rtxn if we actually started it Fixes Coverity 184960
2018-09-29Merge pull request #4459Riccardo Spagni1-5/+16
bcf3f6af fuzz_tests: catch unhandled exceptions (moneromooo-monero) 3ebd05d4 miner: restore stream flags after changing them (moneromooo-monero) a093092e levin_protocol_handler_async: do not propagate exception through dtor (moneromooo-monero) 1eebb82b net_helper: do not propagate exceptions through dtor (moneromooo-monero) fb6a3630 miner: do not propagate exceptions through dtor (moneromooo-monero) 2e2139ff epee: do not propagate exception through dtor (moneromooo-monero) 0749a8bd db_lmdb: do not propagate exceptions in dtor (moneromooo-monero) 1b0afeeb wallet_rpc_server: exit cleanly on unhandled exceptions (moneromooo-monero) 418a9936 unit_tests: catch unhandled exceptions (moneromooo-monero) ea7f9543 threadpool: do not propagate exceptions through the dtor (moneromooo-monero) 6e855422 gen_multisig: nice exit on unhandled exception (moneromooo-monero) 53df2deb db_lmdb: catch error in mdb_stat calls during migration (moneromooo-monero) e67016dd blockchain_blackball: catch failure to commit db transaction (moneromooo-monero) 661439f4 mlog: don't remove old logs if we failed to rename the current file (moneromooo-monero) 5fdcda50 easylogging++: test for NULL before dereference (moneromooo-monero) 7ece1550 performance_test: fix bad last argument calling add_arg (moneromooo-monero) a085da32 unit_tests: add check for page size > 0 before dividing (moneromooo-monero) d8b1ec8b unit_tests: use std::shared_ptr to shut coverity up about leaks (moneromooo-monero) 02563bf4 simplewallet: top level exception catcher to print nicer messages (moneromooo-monero) c57a65b2 blockchain_blackball: fix shift range for 32 bit archs (moneromooo-monero)
2018-09-29Merge pull request #4445Riccardo Spagni1-2/+12
fe125647 Fixup RENAME_DB() macro (Howard Chu)
2018-09-12db_lmdb: do not use base for cumulative distributionmoneromooo-monero1-0/+2
it's confusing and needlessly complicated
2018-09-11v8: per byte fee, pad bulletproofs, fixed 11 ring sizemoneromooo-monero2-17/+22
2018-09-10Merge pull request #4293luigi11111-2/+6
9d65399 is_hdd update (p8p)
2018-09-10Merge pull request #4256luigi11112-7/+3
d6fc870 db_lmdb: resize blockchain database when 90% filled (moneromooo-monero)
2018-09-04Merge pull request #4270luigi11111-2/+13
29dea03 epee: resize vectors where possible in serialization (moneromooo-monero) 76affd9 epee: some speedup in parsing (moneromooo-monero) dc6c069 db_lmdb: speedup the get_output_distribution common case (moneromooo-monero) 76ac5a8 wallet2: ask for a binary output distribution, for speed (moneromooo-monero)
2018-08-25is_hdd updatep8p1-2/+6
2018-08-22Merge pull request #4200luigi11112-50/+0
43f7110 blockchain_db: remove unused get_output_key variant (moneromooo-monero)
2018-08-19db_lmdb: speedup the get_output_distribution common casemoneromooo-monero1-2/+13
2018-08-15Merge pull request #4161luigi11111-2/+2
be02eb9 db_lmdb: demote the 'batch already enabled' log, it's harmless (moneromooo-monero)
2018-08-14db_lmdb: resize blockchain database when 90% filledmoneromooo-monero2-7/+3
instead of a random ratio from 60% to 90%.
2018-07-30blockchain_db: remove unused get_output_key variantmoneromooo-monero2-50/+0
It was actually incorrect, as it would not return commitment
2018-07-27Merge pull request #4129luigi11111-0/+3
ee31383 db_lmdb: don't sync a read only DB (moneromooo-monero)
2018-07-20db_lmdb: demote the "batch already enabled" log, it's harmlessmoneromooo-monero1-2/+2
2018-07-19Merge pull request #4013luigi11112-0/+13
e5592c4 rpc: add blockchain disk size to getinfo (moneromooo-monero)
2018-07-19Merge pull request #3981luigi11112-6/+211
45e419b db: store cumulative rct output distribution in the db for speed (moneromooo-monero)
2018-07-19Merge pull request #3854luigi11112-1/+18
149da42 db_lmdb: enable batch transactions by default (stoffu) 34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn) 9e1403e update get_info RPC and bump RPC version (vicsn) 207b66e first new functional tests (vicsn)
2018-07-13db: store cumulative rct output distribution in the db for speedmoneromooo-monero2-6/+211
This gets rid of the temporary precalc cache. Also make the RPC able to send data back in binary or JSON, since there can be a lot of data This bumps the LMDB database format to v3, with migration.
2018-07-11db_lmdb: don't sync a read only DBmoneromooo-monero1-0/+3
This would only throw
2018-06-29add --regtest and --fixed-difficulty for regression testingvictorsintnicolaas2-0/+17
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest. Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'. Queries hard fork heights info of other network types
2018-06-20rpc: add blockchain disk size to getinfomoneromooo-monero2-0/+13
This should help new nodes predict how much disk space will be needed for a full sync
2018-06-14db_lmdb: enable batch transactions by defaultstoffu1-1/+1
2018-06-11db_ldmb: print db version when complaining about incompatibilitymoneromooo-monero1-4/+5
2018-06-08db_lmdb: warn about slowness when running off a spinning diskmoneromooo-monero1-0/+3
2018-05-30Merge pull request #3876luigi11111-0/+1
740da1b core: fix automatic safe db sync mode switching (moneromooo-monero) e942d34 protocol: do not switch to unsafe sync mode for just a few blocks (moneromooo-monero)
2018-05-28core: fix automatic safe db sync mode switchingmoneromooo-monero1-0/+1
2018-05-23db_lmdb: save pruned and prunable tx data separatelymoneromooo-monero2-41/+328
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-21speed up get_output_distribution (and precalc common case)moneromooo-monero2-0/+43
2018-04-23speedup get_output_histogram for all amounts when min_count > 0moneromooo-monero2-5/+9
This skips the vast majority of "dust" output amounts with just one instance on the chain. Clocks in at 0.15% of the original time on testnet.
2018-03-16add RPC to get a histogram of outputs of a given amountmoneromooo-monero2-3/+44
2018-03-05Merge pull request #3307Riccardo Spagni1-1/+2
223d7d0c db_lmdb: fix free space reporting (moneromooo-monero)
2018-03-05Correct spelling mistakes.Edward Betts1-1/+1
2018-02-23db_lmdb: fix free space reportingmoneromooo-monero1-1/+2
reported by Brad Richards
2018-02-19db_lmdb: fix return code mixup in for_all_*moneromooo-monero1-12/+12
2018-02-16Merge pull request #3226Riccardo Spagni1-4/+7
e4646379 keccak: fix mdlen bounds sanity checking (moneromooo-monero) 2e3e90ac pass large parameters by const ref, not value (moneromooo-monero) 61defd89 blockchain: sanity check number of precomputed hash of hash blocks (moneromooo-monero) 9af6b2d1 ringct: fix infinite loop in unused h2b function (moneromooo-monero) 8cea8d0c simplewallet: double check a new multisig wallet is multisig (moneromooo-monero) 9b98a6ac threadpool: catch exceptions in dtor, to avoid terminate (moneromooo-monero) 24803ed9 blockchain_export: fix buffer overflow in exporter (moneromooo-monero) f3f7da62 perf_timer: rewrite to make it clear there is no division by zero (moneromooo-monero) c6ea3df0 performance_tests: remove add_arg call stray extra param (moneromooo-monero) fa6b4566 fuzz_tests: fix an uninitialized var in setup (moneromooo-monero) 03887f11 keccak: fix sanity check bounds test (moneromooo-monero) ad11db91 blockchain_db: initialize m_open in base class ctor (moneromooo-monero) bece67f9 miner: restore std::cout precision after modification (moneromooo-monero) 1aabd14c db_lmdb: check hard fork info drop succeeded (moneromooo-monero)
2018-02-16Merge pull request #3231Riccardo Spagni2-2/+10
84a8b2da Don't create readtxn until after txn_safe gate check (Howard Chu)
2018-02-02blockchain_db: initialize m_open in base class ctormoneromooo-monero1-2/+1
It's cleaner this way, since it's a base class field Coverity 136568
2018-02-02db_lmdb: check hard fork info drop succeededmoneromooo-monero1-2/+6
Coverity 136364
2018-01-31Don't create readtxn until after txn_safe gate checkHoward Chu2-2/+10
2018-01-26Update 2018 copyrightxmr-eric2-2/+2
2018-01-10Merge pull request #2971Riccardo Spagni2-4/+6
ae860230 Fix exceptions not finding txpool txes when relaying (moneromooo-monero)
2017-12-28Additional fix for core_testsHoward Chu1-10/+15
Reset thread-local info if it doesn't match the current env. Only happens when a process opens/closes env multiple times in the same process, doesn't affect monerod.
2017-12-28Fix stale readcursor flagsHoward Chu1-22/+15
Reset thread-specific flags when a write txn is started. Also remove some redundant start-readtxn code.
2017-12-20Fix exceptions not finding txpool txes when relayingmoneromooo-monero2-4/+6
2017-12-18catch const exceptionsmoneromooo-monero1-1/+1
2017-12-16move includes around to lessen overall loadmoneromooo-monero2-1/+2
2017-11-14Merge pull request #2629Riccardo Spagni1-0/+1
383ff4f6 remove "using namespace std" from headers (moneromooo-monero)
2017-11-14remove "using namespace std" from headersmoneromooo-monero1-0/+1
It's nasty, and actually breaks on Solaris, where if.h fails to build due to: struct map *if_memmap;
2017-11-14Merge pull request #2742Riccardo Spagni1-0/+6
7c7d3672 Increase LMDB maxreaders if large number of threads in use (Howard Chu) 6738753b Use max_concurrency as-is (Howard Chu)
2017-11-08Protect node privacy by proper filtering in restricted-mode RPC answersbinaryFate2-9/+40
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode. This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected. In practice, when running with `--restricted-rpc`: * get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero. * get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions * get_transaction_pool_hashes.bin will not list such transaction * get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality. Fixes #2590.
2017-11-02Increase LMDB maxreaders if large number of threads in useHoward Chu1-0/+6
2017-10-26fixed an LMDB issue on OpenBSD (#2699) which leads to a crash in monerodston1th1-0/+6
2017-10-15Merge pull request #2640Riccardo Spagni1-1/+1
22b51e06 db_lmdb: include chain height when failing to find an output key (moneromooo-monero) 5db433b3 blockchain: avoid exceptions in output verification (moneromooo-monero)
2017-10-12db_lmdb: include chain height when failing to find an output keymoneromooo-monero1-1/+1
To help debugging logs.
2017-10-07frob level 1 logs a bit for consistencymoneromooo-monero1-7/+7
Level 1 logs map to INFO, so setting log level to 1 should show these. Demote some stuff to DEBUG to avoid spam, though.
2017-09-25Merge pull request #2457Riccardo Spagni2-8/+14
d2d8a41c Use actual batch size for resize estimates (Howard Chu)
2017-09-19db_lmdb: fix use of uninitialized key in for_blocks_rangemoneromooo-monero1-2/+2
2017-09-17Use actual batch size for resize estimatesHoward Chu2-8/+14
And optimize import startup: Remember start_height position during initial count_blocks pass to avoid having to reread entire file again to arrive at start_height
2017-08-29Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocksmoneromooo-monero2-9/+25
2017-08-20Toggle SAFE syncmode on and off automaticallyHoward Chu2-0/+7
If monerod is started with default sync mode, set it to SAFE after synchronization completes. Set it back to FAST if synchronization restarts (e.g. because another peer has a longer blockchain). If monerod is started with an explicit sync mode, none of this automation takes effect.
2017-08-19DB cleanupHoward Chu1-1/+11
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp should ever be including DB-specific headers any more.
2017-08-12core: add --db-salvage command line flagmoneromooo-monero1-1/+1
Use to load the database when the primary meta page is corrupted
2017-07-31some include cleanupmoneromooo-monero1-0/+1
2017-06-11blockchain_db: add a txpool tx getter which returns existencemoneromooo-monero2-3/+13
Avoids exception spam for the "nope, not found" case
2017-06-01Don't copy blockchain for coinbase_tx_sumHoward Chu2-3/+13
Changed Blockchain::for_all_blocks() to for_blocks_range() Operate on blockchain in-place instead of building a copy first.
2017-05-25Move txpool to the databasemoneromooo-monero2-0/+235
Integration could go further (ie, return_tx_to_pool calls should not be needed anymore, possibly other things). poolstate.bin is now obsolete.
2017-03-19Partial fix #1876, check thread in block_txn_start()Howard Chu1-1/+2
Don't allow use of existing batch txn if it's from the wrong thread
2017-03-03Remove redundant num_txs() methodHoward Chu2-18/+4
Cleanup of bf1348b7e2b2c72a6d40b23567afaa46b53e6cb7
2017-02-21update copyright year, fix occasional lack of newline at line endRiccardo Spagni2-2/+2
2017-02-21Merge pull request #1727Riccardo Spagni2-28/+9
0288310e blockchain_db: add "raw" blobdata getters for block and transaction (moneromooo-monero)
2017-02-21Merge pull request #1724Riccardo Spagni2-2/+9
cca95c1c blockchain_db: do not throw on expected partial results getting keys (moneromooo-monero)
2017-02-18Handle map resizes from other processesHoward Chu1-9/+52
2017-02-18Can't cache num_txs or num_outputs eitherHoward Chu2-34/+34
Same reason as 3ff54bdd7a8b5e08e4e8ac17b7fff23ad3a82312
2017-02-18Add a few read txns to streamlineHoward Chu2-3/+7
Slight perf gain, but mainly to reduce spam at loglevel 3
2017-02-13blockchain_db: add "raw" blobdata getters for block and transactionmoneromooo-monero2-28/+9
This speeds up operations such as serving blocks to syncing peers
2017-02-13blockchain_db: do not throw on expected partial results getting keysmoneromooo-monero2-2/+9
When scanning for outputs used in a set of incoming blocks, we expect that some of the inputs in their transactions will not be found in the blockchain, as they could be in previous blocks in that set. Those outputs will be scanned there at a later point. In this case, we add a flag to control wehther an output not being found is expected or not.
2017-02-11db_lmdb: fix bad height saved in tx datamoneromooo-monero1-0/+1
The recent change to not keep separate track of the blockchain height caused the reported height to jump early in the lmdb transaction (when the block data is added to the blocks table), rather than at the end, after everything succeeded. Since the block data is added before the transaction data, this caused the transaction data to be saved with a height one more than its expected value. Fix this by saving the block data last. This should have no side effects.
2017-02-08extract some basic code from libcryptonote_core into libcryptonote_basickenshi841-1/+1
2017-01-28Update and use blockchain data files definesMiguel Herranz1-4/+5
2017-01-16Tweak some msg loglevelsHoward Chu1-11/+13
2017-01-16Change logging to easylogging++moneromooo-monero1-17/+21
This replaces the epee and data_loggers logging systems with a single one, and also adds filename:line and explicit severity levels. Categories may be defined, and logging severity set by category (or set of categories). epee style 0-4 log level maps to a sensible severity configuration. Log files now also rotate when reaching 100 MB. To select which logs to output, use the MONERO_LOGS environment variable, with a comma separated list of categories (globs are supported), with their requested severity level after a colon. If a log matches more than one such setting, the last one in the configuration string applies. A few examples: This one is (mostly) silent, only outputting fatal errors: MONERO_LOGS=*:FATAL This one is very verbose: MONERO_LOGS=*:TRACE This one is totally silent (logwise): MONERO_LOGS="" This one outputs all errors and warnings, except for the "verify" category, which prints just fatal errors (the verify category is used for logs about incoming transactions and blocks, and it is expected that some/many will fail to verify, hence we don't want the spam): MONERO_LOGS=*:WARNING,verify:FATAL Log levels are, in decreasing order of priority: FATAL, ERROR, WARNING, INFO, DEBUG, TRACE Subcategories may be added using prefixes and globs. This example will output net.p2p logs at the TRACE level, but all other net* logs only at INFO: MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE Logs which are intended for the user (which Monero was using a lot through epee, but really isn't a nice way to go things) should use the "global" category. There are a few helper macros for using this category, eg: MGINFO("this shows up by default") or MGINFO_RED("this is red"), to try to keep a similar look and feel for now. Existing epee log macros still exist, and map to the new log levels, but since they're used as a "user facing" UI element as much as a logging system, they often don't map well to log severities (ie, a log level 0 log may be an error, or may be something we want the user to see, such as an important info). In those cases, I tried to use the new macros. In other cases, I left the existing macros in. When modifying logs, it is probably best to switch to the new macros with explicit levels. The --log-level options and set_log commands now also accept category settings, in addition to the epee style log levels.
2017-01-15Merge pull request #1506Riccardo Spagni2-18/+41
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu) eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu) c903c554 Don't cache block height, always get from DB (Howard Chu) eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu) 0693cff9 Use batch transactions when syncing (Howard Chu)
2017-01-15Check for correct thread before ending batch transactionHoward Chu1-0/+7
2017-01-14Don't cache block height, always get from DBHoward Chu2-14/+29
2017-01-14Use batch transactions when syncingHoward Chu2-4/+5
Faster throughput while avoiding corruption. I.e., makes running with --db-sync-mode safe more tolerable.
2017-01-07blockchain: allow marking "tx not found" without an exceptionmoneromooo-monero2-3/+13
This is a normal occurence in many cases, and there is no need to spam the log with those when it is.
2016-12-01db_lmdb: add info in an error message when we can't get an outputmoneromooo-monero1-1/+1
Will be useful to debug
2016-12-01db_lmdb: guard against going out of sync on unexpected db resultsmoneromooo-monero1-1/+1
m_num_outputs keeps track of the number of outputs, which should be the same as the size of both the output_txs and output_amounts databases. If one goes out of sync, we need to throw to abort whatever it is we were doing.
2016-12-01db_lmdb: minor pedantic tweaksmoneromooo-monero1-8/+8
Add consts in a few places where it makes sense, avoid unnecessary memory reallocation where we know the full size needed at the outset, simplify and avoid memory copy.
2016-12-01db_lmdb: set same packing format for output_data_t and pre_rct_output_data_tmoneromooo-monero1-0/+3
For safety, though it seems to have been the case already. Also add a comment about the necessary layout identity.
2016-10-15wallet: select part of the fake outs from recent outputsmoneromooo-monero2-11/+28
25% of the outputs are selected from the last 5 days (if possible), in order to avoid the common case of sending recently received outputs again. 25% and 5 days are subject to review later, since it's just a wallet level change.
2016-09-29Merge pull request #1128Riccardo Spagni1-1/+1
d2e11f3 db_lmdb: do not error if dropping a non existent hard fork table (moneromooo-monero)
2016-09-24db_lmdb: do not error if dropping a non existent hard fork tablemoneromooo-monero1-1/+1
2016-09-21Fix minor typo in "need resize" message.Myagui1-2/+2
Message observed while synchronizing a node from scratch. "LMDB memory map needs resized" Proposing a change to: "LMDB memory map needs to be resized"
2016-08-31core: faster find_blockchain_supplementmoneromooo-monero2-2/+9
Since this queries block heights for blocks that may or may not exist, queries for non existing blocks would throw an exception, and that would slow down the loop a lot. 7 seconds to go through a 30 hash list. Fix this by adding an optional return block height to block_exists and using this instead. Actual errors will still throw an exception. This also cuts down on log exception spam.
2016-08-29db_lmdb: fix reorg with "halfway rct" coinbase outputsmoneromooo-monero1-1/+3
2016-08-28move the rct commitments to the output_amounts databasemoneromooo-monero2-128/+53
Since these are needed at the same time as the output pubkeys, this is a whole lot faster, and takes less space. Only outputs of 0 amount store the commitment. When reading other outputs, a fake commitment is regenerated on the fly. This avoids having to rewrite the database to add space for fake commitments for existing outputs. This code relies on two things: - LMDB must support fixed size records per key, rather than per database (ie, all records on key 0 are the same size, all records for non 0 keys are same size, but records from key 0 and non 0 keys do have different sizes). - the commitment must be directly after the rest of the data in outkey and output_data_t.
2016-08-28add rct to the protocolmoneromooo-monero1-0/+2
It is not yet constrained to a fork, so don't use on the real network or you'll be orphaned or rejected.
2016-08-28db_lmdb: update reset for recent db changesmoneromooo-monero1-0/+10
- we need to drop the new m_tx_indices database - we reset the version to current version This fixes the core tests failing to initialize.
2016-08-28blockchain_db: add functions for adding/removing/getting rct commitmentsmoneromooo-monero2-0/+120
2016-08-26db_lmdb: do not try to modify the database in read only modemoneromooo-monero1-4/+12
2016-08-11Merge pull request #948Riccardo Spagni2-6/+24
11dc091 Fake outs set is now decided by the wallet (moneromooo-monero) 1593553 new unlocked parameter to output_histogram (moneromooo-monero)
2016-08-10db_lmdb: errors dropping hf starting height db on reset are not fatalmoneromooo-monero1-2/+1
This db is now dropped unconditionally, so may or may not be there in the first place.
2016-08-01new unlocked parameter to output_histogrammoneromooo-monero2-6/+24
This constrains the number of instances of any amount to the unlocked ones (as defined by the default unlock time setting: outputs with non default unlock time are not considered, so may be counted as unlocked even if they are not actually unlocked).
2016-07-26db_lmdb: add some missing api call checksmoneromooo-monero1-14/+31
2016-07-13remove hf_starting_height dbmoneromooo-monero2-76/+2
It's not really needed, it used to be an optimization for when that code was not using the db and needed to recalculate things fast on startup.
2016-04-24Another take on migrationHoward Chu1-570/+94
Delete old indices and recreate them, rather than updating them Maybe not quite as slow as before.
2016-04-08MigrationHoward Chu2-5/+1049
Migrate from DB version 0 to version 1 on startup
2016-04-05Merge branch 'performance' into masterHoward Chu2-715/+559
2016-04-05CleanupHoward Chu2-21/+65
drop obsolete remove_output() fix get_output_key(global), fix crash in blockchain_dump
2016-04-05Cleanup and clarifyHoward Chu2-115/+92
Try to rationalize the variable names, document usage.
2016-04-05Use cursors in some remove functionsHoward Chu1-7/+33
Helps when they're called repeatedly in one txn
2016-04-05Use DUPFIXED for block_info and output_txsHoward Chu2-56/+46
Saves another ~150MB or so on the full blockchain
2016-04-05More outputs consolidationHoward Chu2-491/+263
Also bumped DB VERSION to 1 Another significant speedup and space savings: Get rid of global_output_indices, remove indirection from output to keys This is the change warptangent described on irc but never got to finish.
2016-04-05Use DUPFIXED for output_keysHoward Chu1-23/+37
Saves another 90MB on 200000 block import. Had to bring back compare_uint64 for this, but it's safe since this table is always 64-bit aligned.
2016-04-05Use DUPFIXED for tx_indicesHoward Chu1-50/+60
Small space savings, no measurable speedup
2016-04-05Use DUPFIXED for block_heightsHoward Chu1-22/+29
Only a small savings...
2016-04-05Use DUPFIXED for spent_keysHoward Chu1-23/+18
2016-04-05Add back changes from revert.warptangent1-19/+2
m_tx_outputs doesn't need to be changed, as it's no longer dup list.
2016-04-05Use MDB_APPEND mode with two tx subdbswarptangent1-2/+2
This is possible on those using a tx index as a key.
2016-04-05Schema update: tx_indices - consolidate the tx subdbs from 5 to 3warptangent2-75/+47
2016-04-05Schema update: tx_indices - yet less indirectionwarptangent2-16/+11
2016-04-05Schema update: tx_indices - improve further with less indirectionwarptangent2-17/+50
2016-04-05Add batch warning for further reviewwarptangent1-0/+4
2016-04-05Schema update: tx_indiceswarptangent2-32/+109
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-20 17:45:27 +0000