aboutsummaryrefslogtreecommitdiff
path: root/src/blockchain_db (follow)
AgeCommit message (Collapse)AuthorFilesLines
2019-07-24Merge pull request #5524luigi11114-1/+239
06b8f29 blockchain: keep alternative blocks in LMDB (moneromooo-monero)
2019-07-24Merge pull request #5502luigi11111-1/+1
25a7cfd add a few checks where it seems appropriate (moneromooo-monero) 1a66a86 remove unused code (moneromooo-monero)
2019-06-15db_lmdb: commit pruning txn at checkpointsmoneromooo-monero1-1/+50
to avoid errors when the txn is too large
2019-06-14ensure no NULL is passed to memcpymoneromooo-monero1-2/+2
NULL is valid when size is 0, but memcpy uses nonnull attributes, so let's not poke the bear
2019-05-25core: do not commit half constructed batch db txnmoneromooo-monero2-0/+16
2019-05-08blockchain: keep alternative blocks in LMDBmoneromooo-monero4-1/+239
Alternative blocks are cleared on startup unless --keep-alt-blocks is passed on the command line
2019-05-01fix wide difficulty conversion with some versions of boostmoneromooo-monero1-2/+2
2019-04-29add a few checks where it seems appropriatemoneromooo-monero1-1/+1
2019-04-17testdb: add override keyword where missingstoffu3-132/+100
and delete obsolete BlockchainBDB::get_tx_output_indices along the way
2019-04-14blockchain_db: fix db txn ending too earlymoneromooo-monero5-58/+98
The db txn in add_block ending caused the entire overarching batch txn to stop. Also add a new guard class so a db txn can be stopped in the face of exceptions. Also use a read only db txn in init when the db itself is read only, and do not save the max tx size in that case.
2019-04-05cryptonote: rework block blob size sanity checkmoneromooo-monero4-0/+72
Use the actual block weight limit, assuming that weight is always greater or equal to size
2019-03-24Make difficulty 128 bit instead of 64 bitmoneromooo-monero2-7/+155
Based on Boolberry work by: jahrsg <jahr@jahr.me> cr.zoidberg <crypto.zoidberg@gmail.com>
2019-03-21Merge pull request #5256Riccardo Spagni4-0/+94
4b21d38d blockchain: speed up getting N blocks weights/long term weights (moneromooo-monero)
2019-03-08blockchain: speed up getting N blocks weights/long term weightsmoneromooo-monero4-0/+94
2019-03-05Update 2019 copyrightbinaryFate9-9/+9
2019-03-05Merge pull request #5119Riccardo Spagni1-5/+5
f024a10b db_lmdb: make mdb_block_info format conversion more future proof (moneromooo-monero)
2019-03-05db_lmdb: make mdb_block_info format conversion more future proofmoneromooo-monero1-5/+5
If mdb_block_info changes again, the v2 to v3 conversion would convert to an incorrect format.
2019-03-05save some database calls when getting top block hash and heightmoneromooo-monero4-4/+8
2019-03-05Avoid repeated (de)serialization when syncingmoneromooo-monero5-22/+27
2019-03-05db: speedup block additionmoneromooo-monero1-11/+16
by avoiding repeated (de)serialization
2019-03-04Merge pull request #5081Riccardo Spagni1-3/+3
d78addcb db_lmdb: don't unnecessarily cast to double on the way to uint64_t (moneromooo-monero)
2019-03-04Merge pull request #5175Riccardo Spagni1-0/+1
85088d9f db_lmdb: fix missing mdb_dbi_close in migration (moneromooo-monero)
2019-03-04ArticMine's new block weight algorithmmoneromooo-monero5-8/+384
This curbs runaway growth while still allowing substantial spikes in block weight Original specification from ArticMine: here is the scaling proposal Define: LongTermBlockWeight Before fork: LongTermBlockWeight = BlockWeight At or after fork: LongTermBlockWeight = min(BlockWeight, 1.4*LongTermEffectiveMedianBlockWeight) Note: To avoid possible consensus issues over rounding the LongTermBlockWeight for a given block should be calculated to the nearest byte, and stored as a integer in the block itself. The stored LongTermBlockWeight is then used for future calculations of the LongTermEffectiveMedianBlockWeight and not recalculated each time. Define: LongTermEffectiveMedianBlockWeight LongTermEffectiveMedianBlockWeight = max(300000, MedianOverPrevious100000Blocks(LongTermBlockWeight)) Change Definition of EffectiveMedianBlockWeight From (current definition) EffectiveMedianBlockWeight = max(300000, MedianOverPrevious100Blocks(BlockWeight)) To (proposed definition) EffectiveMedianBlockWeight = min(max(300000, MedianOverPrevious100Blocks(BlockWeight)), 50*LongTermEffectiveMedianBlockWeight) Notes: 1) There are no other changes to the existing penalty formula, median calculation, fees etc. 2) There is the requirement to store the LongTermBlockWeight of a block unencrypted in the block itself. This is to avoid possible consensus issues over rounding and also to prevent the calculations from becoming unwieldy as we move away from the fork. 3) When the EffectiveMedianBlockWeight cap is reached it is still possible to mine blocks up to 2x the EffectiveMedianBlockWeight by paying the corresponding penalty. Note: the long term block weight is stored in the database, but not in the actual block itself, since it requires recalculating anyway for verification.
2019-02-21db_lmdb: fix missing mdb_dbi_close in migrationmoneromooo-monero1-0/+1
Fixed by hyc
2019-01-22Pruningmoneromooo-monero4-25/+491
The blockchain prunes seven eighths of prunable tx data. This saves about two thirds of the blockchain size, while keeping the node useful as a sync source for an eighth of the blockchain. No other data is currently pruned. There are three ways to prune a blockchain: - run monerod with --prune-blockchain - run "prune_blockchain" in the monerod console - run the monero-blockchain-prune utility The first two will prune in place. Due to how LMDB works, this will not reduce the blockchain size on disk. Instead, it will mark parts of the file as free, so that future data will use that free space, causing the file to not grow until free space grows scarce. The third way will create a second database, a pruned copy of the original one. Since this is a new file, this one will be smaller than the original one. Once the database is pruned, it will stay pruned as it syncs. That is, there is no need to use --prune-blockchain again, etc.
2019-01-19db_lmdb: don't unnecessarily cast to double on the way to uint64_tmoneromooo-monero1-3/+3
2019-01-16blockchain_db: allow getting output keys without commitmentmoneromooo-monero3-4/+5
Since the commitment has to be calculated for non rct outputs, it slows down a lot unnecessarily if we don't need it
2019-01-16Merge pull request #4984Riccardo Spagni3-23/+30
008647d7 blockchain_db: speedup tx output gathering (moneromooo-monero)
2019-01-07Make get_output_key method constmoneroexamples4-8/+8
get_output_key method is commonly used when working with txs and their key images. Because the method is not const, passing blockchain object though const& or pointers to const is not possible in this context. This is especially problematic in external projects (e.g., projects in moneroexamples) that use monero C++ api to operate on the blockchain and txs. Thus, having get_output_key method will simplify moving blockchain object around through const references and pointers to const objects.
2018-12-31Merge pull request #4946luigi11115-96/+13
6644b9b blockchain_db: remove a couple unused functions (moneromooo-monero) ce594f5 blockchain_db: allocate known size vector only once (moneromooo-monero) 8332698 db_lmdb: inline check_open, it's trivial and called everywhere (moneromooo-monero) 5511563 db_lmdb: avoid pointless division (moneromooo-monero) d1efe3d cryptonote: set tx hash on newly parsed txes when known (moneromooo-monero) 9cc68a2 tx_pool: add a few std::move where it can make a difference (moneromooo-monero)
2018-12-18blockchain_db: speedup tx output gatheringmoneromooo-monero3-23/+30
We know all the data we'll want for getblocks.bin is contiguous
2018-12-12Merge pull request #4901Riccardo Spagni3-4/+75
a48f2dab blockchain_prune_known_spent_data: blackball file is now optional (moneromooo-monero) 17b45725 Outputs where all amounts are known spent can now be pruned (moneromooo-monero)
2018-12-05db_lmdb: avoid pointless divisionmoneromooo-monero1-1/+1
2018-12-05db_lmdb: inline check_open, it's trivial and called everywheremoneromooo-monero2-8/+7
2018-12-05blockchain_db: allocate known size vector only oncemoneromooo-monero1-5/+5
2018-12-05blockchain_db: remove a couple unused functionsmoneromooo-monero4-82/+0
2018-12-04Merge pull request #4872Riccardo Spagni1-10/+6
fc99f177 lmdb: fix gcc 7.3.0 'implicit-fallthrough' warning (xiphon)
2018-12-04Merge pull request #4854Riccardo Spagni2-3/+0
bd98e99c Removed a lot of unnecessary includes (Martijn Otto)
2018-11-27Outputs where all amounts are known spent can now be prunedmoneromooo-monero3-4/+75
Only for pre rct for obvious reasons. Note: DO NOT use a known spent list which includes outputs which are not known spent. If the list includes any output that's just strongly thought to be spent, but not provably so, you risk finding yourself unable to sync past the point where that output is spent. I estimate only 200 MB saved on current mainnet though, unless the new blackballing rule unearths a good amount of large-amount-set extra spent outs.
2018-11-26Merge pull request #4821Riccardo Spagni3-8/+12
fc98f7a0 rpc: speedup get_outs.bin (moneromooo-monero)
2018-11-26rpc: speedup get_outs.binmoneromooo-monero3-8/+12
2018-11-26Merge pull request #4830Riccardo Spagni1-0/+9
dffec258 db_lmdb: error out if the db needs migration in read only mode (moneromooo-monero)
2018-11-19lmdb: fix gcc 7.3.0 'implicit-fallthrough' warningxiphon1-10/+6
2018-11-15Removed a lot of unnecessary includesMartijn Otto2-3/+0
2018-11-14Merge pull request #4769Riccardo Spagni3-22/+27
5808530f blockchain: remove unused output_scan_worker parameter (moneromooo-monero) 1426209a blockchain: don't run threads if we have just one function to run (moneromooo-monero) 6f7a5fd4 db_lmdb: slight speedup getting array data from the blockchain (moneromooo-monero) 99fbe100 db_lmdb: save some string copies for readonly db keys/values (moneromooo-monero) bf31447e tx_pool: speed up take_tx for transactions from blocks (moneromooo-monero) 4f005a77 tx_pool: remove unnecessary get_transaction_hash (moneromooo-monero) 593ef598 perf_timer: call reserve on new timer array (moneromooo-monero) 6ecc99ad core: avoid unnecessary tx/blob conversions (moneromooo-monero) 00cc1a16 unit_tests: notify test special case for the usual weirdo (moneromooo-monero)
2018-11-09db_lmdb: error out if the db needs migration in read only modemoneromooo-monero1-0/+9
2018-11-04db_lmdb: slight speedup getting array data from the blockchainmoneromooo-monero1-6/+8
2018-11-02db_lmdb: log which output was not found in get_output_keymoneromooo-monero1-1/+2
2018-11-01Merge pull request #4664Riccardo Spagni1-9/+23
61304151 db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedup (moneromooo-monero)
2018-11-01Merge pull request #4639Riccardo Spagni1-1/+1
ec301390 Update db_lmdb.cpp (Cactii1)
2018-11-01db_lmdb: save some string copies for readonly db keys/valuesmoneromooo-monero1-11/+16
2018-11-01core: avoid unnecessary tx/blob conversionsmoneromooo-monero3-6/+4
2018-10-19db_lmdb: use MDB_MULTIPLE_NEXT where possible for some speedupmoneromooo-monero1-9/+23
2018-10-17Update db_lmdb.cppCactii11-1/+1
2018-10-16blockchain_db: initialize m_hardfork in ctor just in casemoneromooo-monero1-1/+1
Coverity 136568
2018-10-02Fix rtxn usage in BlockchainLMDB::get_estimated_batch_sizeHoward Chu1-2/+2
Should only stop the rtxn if we actually started it Fixes Coverity 184960
2018-09-29Merge pull request #4459Riccardo Spagni1-5/+16
bcf3f6af fuzz_tests: catch unhandled exceptions (moneromooo-monero) 3ebd05d4 miner: restore stream flags after changing them (moneromooo-monero) a093092e levin_protocol_handler_async: do not propagate exception through dtor (moneromooo-monero) 1eebb82b net_helper: do not propagate exceptions through dtor (moneromooo-monero) fb6a3630 miner: do not propagate exceptions through dtor (moneromooo-monero) 2e2139ff epee: do not propagate exception through dtor (moneromooo-monero) 0749a8bd db_lmdb: do not propagate exceptions in dtor (moneromooo-monero) 1b0afeeb wallet_rpc_server: exit cleanly on unhandled exceptions (moneromooo-monero) 418a9936 unit_tests: catch unhandled exceptions (moneromooo-monero) ea7f9543 threadpool: do not propagate exceptions through the dtor (moneromooo-monero) 6e855422 gen_multisig: nice exit on unhandled exception (moneromooo-monero) 53df2deb db_lmdb: catch error in mdb_stat calls during migration (moneromooo-monero) e67016dd blockchain_blackball: catch failure to commit db transaction (moneromooo-monero) 661439f4 mlog: don't remove old logs if we failed to rename the current file (moneromooo-monero) 5fdcda50 easylogging++: test for NULL before dereference (moneromooo-monero) 7ece1550 performance_test: fix bad last argument calling add_arg (moneromooo-monero) a085da32 unit_tests: add check for page size > 0 before dividing (moneromooo-monero) d8b1ec8b unit_tests: use std::shared_ptr to shut coverity up about leaks (moneromooo-monero) 02563bf4 simplewallet: top level exception catcher to print nicer messages (moneromooo-monero) c57a65b2 blockchain_blackball: fix shift range for 32 bit archs (moneromooo-monero)
2018-09-29Merge pull request #4445Riccardo Spagni1-2/+12
fe125647 Fixup RENAME_DB() macro (Howard Chu)
2018-09-12db_lmdb: do not use base for cumulative distributionmoneromooo-monero1-0/+2
it's confusing and needlessly complicated
2018-09-11v8: per byte fee, pad bulletproofs, fixed 11 ring sizemoneromooo-monero6-37/+42
2018-09-10Merge pull request #4293luigi11111-2/+6
9d65399 is_hdd update (p8p)
2018-09-10Merge pull request #4256luigi11112-7/+3
d6fc870 db_lmdb: resize blockchain database when 90% filled (moneromooo-monero)
2018-09-04Merge pull request #4270luigi11111-2/+13
29dea03 epee: resize vectors where possible in serialization (moneromooo-monero) 76affd9 epee: some speedup in parsing (moneromooo-monero) dc6c069 db_lmdb: speedup the get_output_distribution common case (moneromooo-monero) 76ac5a8 wallet2: ask for a binary output distribution, for speed (moneromooo-monero)
2018-08-25is_hdd updatep8p1-2/+6
2018-08-22Merge pull request #4204luigi11111-2/+2
b278b83 core: sync database based on bytes added, not blocks added (moneromooo-monero)
2018-08-22Merge pull request #4200luigi11114-68/+1
43f7110 blockchain_db: remove unused get_output_key variant (moneromooo-monero)
2018-08-19db_lmdb: speedup the get_output_distribution common casemoneromooo-monero1-2/+13
2018-08-15Merge pull request #4161luigi11111-2/+2
be02eb9 db_lmdb: demote the 'batch already enabled' log, it's harmless (moneromooo-monero)
2018-08-14db_lmdb: resize blockchain database when 90% filledmoneromooo-monero2-7/+3
instead of a random ratio from 60% to 90%.
2018-08-12core: sync database based on bytes added, not blocks addedmoneromooo-monero1-2/+2
Blocks have a very wide range, whereas actual size is the relevant quantity to consider when syncing
2018-07-30blockchain_db: remove unused get_output_key variantmoneromooo-monero4-68/+1
It was actually incorrect, as it would not return commitment
2018-07-27Merge pull request #4129luigi11111-0/+3
ee31383 db_lmdb: don't sync a read only DB (moneromooo-monero)
2018-07-20db_lmdb: demote the "batch already enabled" log, it's harmlessmoneromooo-monero1-2/+2
2018-07-19Merge pull request #4013luigi11113-0/+20
e5592c4 rpc: add blockchain disk size to getinfo (moneromooo-monero)
2018-07-19Merge pull request #3981luigi11114-7/+236
45e419b db: store cumulative rct output distribution in the db for speed (moneromooo-monero)
2018-07-19Merge pull request #3854luigi11115-1/+39
149da42 db_lmdb: enable batch transactions by default (stoffu) 34cb6b4 add --regtest and --fixed-difficulty for regression testing (vicsn) 9e1403e update get_info RPC and bump RPC version (vicsn) 207b66e first new functional tests (vicsn)
2018-07-13db: store cumulative rct output distribution in the db for speedmoneromooo-monero4-7/+236
This gets rid of the temporary precalc cache. Also make the RPC able to send data back in binary or JSON, since there can be a lot of data This bumps the LMDB database format to v3, with migration.
2018-07-11db_lmdb: don't sync a read only DBmoneromooo-monero1-0/+3
This would only throw
2018-06-29add --regtest and --fixed-difficulty for regression testingvictorsintnicolaas5-0/+38
on_generateblocks RPC call combines functionality from the on_getblocktemplate and on_submitblock RPC calls to allow rapid block creation. Difficulty is set permanently to 1 for regtest. Makes use of FAKECHAIN network type, but takes hard fork heights from mainchain Default reserve_size in generate_blocks RPC call is now 1. If it is 0, the following error occurs 'Failed to calculate offset for'. Queries hard fork heights info of other network types
2018-06-27Merge pull request #3987luigi11111-4/+5
2d10830 db_ldmb: print db version when complaining about incompatibility (moneromooo-monero)
2018-06-25Merge pull request #3936luigi11111-0/+1
d81e042 tx_pool: initialize bitflags padding since it gets written to storage (moneromooo-monero)
2018-06-20rpc: add blockchain disk size to getinfomoneromooo-monero3-0/+20
This should help new nodes predict how much disk space will be needed for a full sync
2018-06-14db_lmdb: enable batch transactions by defaultstoffu1-1/+1
2018-06-11db_ldmb: print db version when complaining about incompatibilitymoneromooo-monero1-4/+5
2018-06-08db_lmdb: warn about slowness when running off a spinning diskmoneromooo-monero1-0/+3
2018-06-05tx_pool: initialize bitflags padding since it gets written to storagemoneromooo-monero1-0/+1
Avoids valgrind reporting uninitialized data usage
2018-05-30Merge pull request #3876luigi11111-0/+1
740da1b core: fix automatic safe db sync mode switching (moneromooo-monero) e942d34 protocol: do not switch to unsafe sync mode for just a few blocks (moneromooo-monero)
2018-05-28core: fix automatic safe db sync mode switchingmoneromooo-monero1-0/+1
2018-05-23db_lmdb: save pruned and prunable tx data separatelymoneromooo-monero6-51/+375
This bumps DB version to 2, migration code will run for v1 DBs
2018-05-21speed up get_output_distribution (and precalc common case)moneromooo-monero3-0/+45
2018-04-23speedup get_output_histogram for all amounts when min_count > 0moneromooo-monero3-6/+11
This skips the vast majority of "dust" output amounts with just one instance on the chain. Clocks in at 0.15% of the original time on testnet.
2018-03-16add RPC to get a histogram of outputs of a given amountmoneromooo-monero3-4/+47
2018-03-05Merge pull request #3307Riccardo Spagni1-1/+2
223d7d0c db_lmdb: fix free space reporting (moneromooo-monero)
2018-03-05Correct spelling mistakes.Edward Betts1-1/+1
2018-02-23db_lmdb: fix free space reportingmoneromooo-monero1-1/+2
reported by Brad Richards
2018-02-19db_lmdb: fix return code mixup in for_all_*moneromooo-monero1-12/+12
2018-02-16Merge pull request #3226Riccardo Spagni3-5/+13
e4646379 keccak: fix mdlen bounds sanity checking (moneromooo-monero) 2e3e90ac pass large parameters by const ref, not value (moneromooo-monero) 61defd89 blockchain: sanity check number of precomputed hash of hash blocks (moneromooo-monero) 9af6b2d1 ringct: fix infinite loop in unused h2b function (moneromooo-monero) 8cea8d0c simplewallet: double check a new multisig wallet is multisig (moneromooo-monero) 9b98a6ac threadpool: catch exceptions in dtor, to avoid terminate (moneromooo-monero) 24803ed9 blockchain_export: fix buffer overflow in exporter (moneromooo-monero) f3f7da62 perf_timer: rewrite to make it clear there is no division by zero (moneromooo-monero) c6ea3df0 performance_tests: remove add_arg call stray extra param (moneromooo-monero) fa6b4566 fuzz_tests: fix an uninitialized var in setup (moneromooo-monero) 03887f11 keccak: fix sanity check bounds test (moneromooo-monero) ad11db91 blockchain_db: initialize m_open in base class ctor (moneromooo-monero) bece67f9 miner: restore std::cout precision after modification (moneromooo-monero) 1aabd14c db_lmdb: check hard fork info drop succeeded (moneromooo-monero)
2018-02-16Merge pull request #3231Riccardo Spagni2-2/+10
84a8b2da Don't create readtxn until after txn_safe gate check (Howard Chu)
2018-02-02blockchain_db: initialize m_open in base class ctormoneromooo-monero3-3/+7
It's cleaner this way, since it's a base class field Coverity 136568
2018-02-02db_lmdb: check hard fork info drop succeededmoneromooo-monero1-2/+6
Coverity 136364
2018-01-31Don't create readtxn until after txn_safe gate checkHoward Chu2-2/+10
2018-01-26Update 2018 copyrightxmr-eric8-8/+8
2018-01-10Merge pull request #2971Riccardo Spagni3-6/+9
ae860230 Fix exceptions not finding txpool txes when relaying (moneromooo-monero)
2017-12-28Additional fix for core_testsHoward Chu1-10/+15
Reset thread-local info if it doesn't match the current env. Only happens when a process opens/closes env multiple times in the same process, doesn't affect monerod.
2017-12-28Fix stale readcursor flagsHoward Chu1-22/+15
Reset thread-specific flags when a write txn is started. Also remove some redundant start-readtxn code.
2017-12-20Fix exceptions not finding txpool txes when relayingmoneromooo-monero3-6/+9
2017-12-18catch const exceptionsmoneromooo-monero1-1/+1
2017-12-18fix a few leaks by throwing objects, not newed pointers to objectsmoneromooo-monero1-5/+5
2017-12-16move includes around to lessen overall loadmoneromooo-monero5-3/+5
2017-11-15blockchain_db: sanity check on tx/hash vector sizesflozilla1-0/+4
It could trip on a corrupt/crafted file if the user has disabled input verification.
2017-11-14Merge pull request #2629Riccardo Spagni2-1/+2
383ff4f6 remove "using namespace std" from headers (moneromooo-monero)
2017-11-14remove "using namespace std" from headersmoneromooo-monero2-1/+2
It's nasty, and actually breaks on Solaris, where if.h fails to build due to: struct map *if_memmap;
2017-11-14Merge pull request #2742Riccardo Spagni1-0/+6
7c7d3672 Increase LMDB maxreaders if large number of threads in use (Howard Chu) 6738753b Use max_concurrency as-is (Howard Chu)
2017-11-14Merge pull request #2615Riccardo Spagni3-11/+42
10013e94 Protect node privacy by proper filtering in restricted-mode RPC answers (binaryFate)
2017-11-08Protect node privacy by proper filtering in restricted-mode RPC answersbinaryFate3-11/+42
This patch allows to filter out sensitive information for queries that rely on the pool state, when running in restricted mode. This filtering is only applied to data sent back to RPC queries. Results of inline commands typed locally in the daemon are not affected. In practice, when running with `--restricted-rpc`: * get_transaction_pool will list relayed transactions with the fields "last relayed time" and "received time" set to zero. * get_transaction_pool will not list transaction that have do_not_relay set to true, and will not list key images that are used only for such transactions * get_transaction_pool_hashes.bin will not list such transaction * get_transaction_pool_stats will not count such transactions in any of the aggregated values that are computed The implementation does not make filtering the default, so developers should be mindful of this if they add new RPC functionality. Fixes #2590.
2017-11-06track double spending in the txpoolmoneromooo-monero1-1/+2
Transactions in the txpool are marked when another transaction is seen double spending one or more of its inputs. This is then exposed wherever appropriate. Note that being marked with this "double spend seen" flag does NOT mean this transaction IS a double spend and will never be mined: it just means that the network has seen at least another transaction spending at least one of the same inputs, so care should be taken to wait for a few confirmations before acting upon that transaction (ie, mostly of use for merchants wanting to accept unconfirmed transactions).
2017-11-02Increase LMDB maxreaders if large number of threads in useHoward Chu1-0/+6
2017-10-26fixed an LMDB issue on OpenBSD (#2699) which leads to a crash in monerodston1th1-0/+6
2017-10-15Merge pull request #2640Riccardo Spagni1-1/+1
22b51e06 db_lmdb: include chain height when failing to find an output key (moneromooo-monero) 5db433b3 blockchain: avoid exceptions in output verification (moneromooo-monero)
2017-10-12db_lmdb: include chain height when failing to find an output keymoneromooo-monero1-1/+1
To help debugging logs.
2017-10-07frob level 1 logs a bit for consistencymoneromooo-monero1-7/+7
Level 1 logs map to INFO, so setting log level to 1 should show these. Demote some stuff to DEBUG to avoid spam, though.
2017-09-25Merge pull request #2457Riccardo Spagni3-9/+15
d2d8a41c Use actual batch size for resize estimates (Howard Chu)
2017-09-19db_lmdb: fix use of uninitialized key in for_blocks_rangemoneromooo-monero1-2/+2
2017-09-17Use actual batch size for resize estimatesHoward Chu3-9/+15
And optimize import startup: Remember start_height position during initial count_blocks pass to avoid having to reread entire file again to arrive at start_height
2017-09-02Merge pull request #2372Riccardo Spagni2-9/+25
c867357a cryptonote_protocol: error handling on cleanup_handle_incoming_blocks (moneromooo-monero) ce901fcb Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocks (moneromooo-monero) 84fa015e core: guard against exceptions in handle_incoming_{block,tx} (moneromooo-monero)
2017-08-29Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocksmoneromooo-monero2-9/+25
2017-08-29move db specific options to BlockchainDBmoneromooo-monero2-0/+35
Avoids common depending on blockchain_db, which can cause link errors.
2017-08-26Merge pull request #2314Riccardo Spagni3-0/+14
c22d22e2 Cleanup test impact of adding safesyncmode() method (Howard Chu) 9a859844 Toggle SAFE syncmode on and off automatically (Howard Chu)
2017-08-20Toggle SAFE syncmode on and off automaticallyHoward Chu3-0/+14
If monerod is started with default sync mode, set it to SAFE after synchronization completes. Set it back to FAST if synchronization restarts (e.g. because another peer has a longer blockchain). If monerod is started with an explicit sync mode, none of this automation takes effect.
2017-08-20More DB support cleanupHoward Chu2-5/+34
Hide DB types from db_types.h - no reason to recompile dependencies when DB types change. Also remove lingering in-memory DB references, they've been obsolete since 9e82b694da120708652871b55f639d1ef306a7ec
2017-08-19DB cleanupHoward Chu3-1/+34
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp should ever be including DB-specific headers any more.
2017-08-12core: add --db-salvage command line flagmoneromooo-monero1-1/+1
Use to load the database when the primary meta page is corrupted
2017-07-31some include cleanupmoneromooo-monero2-0/+2
2017-06-11blockchain_db: add a txpool tx getter which returns existencemoneromooo-monero3-3/+23
Avoids exception spam for the "nope, not found" case
2017-06-01Don't copy blockchain for coinbase_tx_sumHoward Chu3-7/+19
Changed Blockchain::for_all_blocks() to for_blocks_range() Operate on blockchain in-place instead of building a copy first.
2017-05-30Merge pull request #2039Riccardo Spagni1-1/+1
4b932ff3 changed crypto to cncrypto so it generated libcncrypto (Gentian)
2017-05-25Move txpool to the databasemoneromooo-monero3-0/+321
Integration could go further (ie, return_tx_to_pool calls should not be needed anymore, possibly other things). poolstate.bin is now obsolete.
2017-05-23changed crypto to cncrypto so it generated libcncryptoGentian1-1/+1
fix a cmakelist
2017-03-19Partial fix #1876, check thread in block_txn_start()Howard Chu1-1/+2
Don't allow use of existing batch txn if it's from the wrong thread
2017-03-10fix dependency: put HardFork back to cryptonote_basic, made some ↵kenshi841-5/+5
BlockchainDB functions virtual again to avoid missing symbols error
2017-03-03Remove redundant num_txs() methodHoward Chu2-18/+4
Cleanup of bf1348b7e2b2c72a6d40b23567afaa46b53e6cb7
2017-02-25core: move hardfork back to cryptonote_coremoneromooo-monero1-1/+1
should fix a cross dependency betewen cryptonote_basic and blockchain_db
2017-02-21update copyright year, fix occasional lack of newline at line endRiccardo Spagni8-8/+8
2017-02-21Merge pull request #1727Riccardo Spagni4-36/+90
0288310e blockchain_db: add "raw" blobdata getters for block and transaction (moneromooo-monero)
2017-02-21Merge pull request #1724Riccardo Spagni3-3/+10
cca95c1c blockchain_db: do not throw on expected partial results getting keys (moneromooo-monero)
2017-02-18Handle map resizes from other processesHoward Chu1-9/+52
2017-02-18Can't cache num_txs or num_outputs eitherHoward Chu2-34/+34
Same reason as 3ff54bdd7a8b5e08e4e8ac17b7fff23ad3a82312
2017-02-18Add a few read txns to streamlineHoward Chu2-3/+7
Slight perf gain, but mainly to reduce spam at loglevel 3
2017-02-13blockchain_db: add "raw" blobdata getters for block and transactionmoneromooo-monero4-36/+90
This speeds up operations such as serving blocks to syncing peers
2017-02-13blockchain_db: do not throw on expected partial results getting keysmoneromooo-monero3-3/+10
When scanning for outputs used in a set of incoming blocks, we expect that some of the inputs in their transactions will not be found in the blockchain, as they could be in previous blocks in that set. Those outputs will be scanned there at a later point. In this case, we add a flag to control wehther an output not being found is expected or not.
2017-02-11db_lmdb: fix bad height saved in tx datamoneromooo-monero2-6/+7
The recent change to not keep separate track of the blockchain height caused the reported height to jump early in the lmdb transaction (when the block data is added to the blocks table), rather than at the end, after everything succeeded. Since the block data is added before the transaction data, this caused the transaction data to be saved with a height one more than its expected value. Fix this by saving the block data last. This should have no side effects.
2017-02-08extract some basic code from libcryptonote_core into libcryptonote_basickenshi845-6/+7
2017-01-28Update and use blockchain data files definesMiguel Herranz1-4/+5
2017-01-16Tweak some msg loglevelsHoward Chu1-11/+13
2017-01-16Fix PR#1506, off by one in chain heightHoward Chu1-3/+2
2017-01-16Change logging to easylogging++moneromooo-monero2-17/+24
This replaces the epee and data_loggers logging systems with a single one, and also adds filename:line and explicit severity levels. Categories may be defined, and logging severity set by category (or set of categories). epee style 0-4 log level maps to a sensible severity configuration. Log files now also rotate when reaching 100 MB. To select which logs to output, use the MONERO_LOGS environment variable, with a comma separated list of categories (globs are supported), with their requested severity level after a colon. If a log matches more than one such setting, the last one in the configuration string applies. A few examples: This one is (mostly) silent, only outputting fatal errors: MONERO_LOGS=*:FATAL This one is very verbose: MONERO_LOGS=*:TRACE This one is totally silent (logwise): MONERO_LOGS="" This one outputs all errors and warnings, except for the "verify" category, which prints just fatal errors (the verify category is used for logs about incoming transactions and blocks, and it is expected that some/many will fail to verify, hence we don't want the spam): MONERO_LOGS=*:WARNING,verify:FATAL Log levels are, in decreasing order of priority: FATAL, ERROR, WARNING, INFO, DEBUG, TRACE Subcategories may be added using prefixes and globs. This example will output net.p2p logs at the TRACE level, but all other net* logs only at INFO: MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE Logs which are intended for the user (which Monero was using a lot through epee, but really isn't a nice way to go things) should use the "global" category. There are a few helper macros for using this category, eg: MGINFO("this shows up by default") or MGINFO_RED("this is red"), to try to keep a similar look and feel for now. Existing epee log macros still exist, and map to the new log levels, but since they're used as a "user facing" UI element as much as a logging system, they often don't map well to log severities (ie, a log level 0 log may be an error, or may be something we want the user to see, such as an important info). In those cases, I tried to use the new macros. In other cases, I left the existing macros in. When modifying logs, it is probably best to switch to the new macros with explicit levels. The --log-level options and set_log commands now also accept category settings, in addition to the epee style log levels.
2017-01-15Merge pull request #1506Riccardo Spagni5-24/+49
3ff54bdd Check for correct thread before ending batch transaction (Howard Chu) eaf8470b Must wait for previous batch to finish before starting new one (Howard Chu) c903c554 Don't cache block height, always get from DB (Howard Chu) eb1fb601 Tweak default db-sync-mode to fast:async:1 (Howard Chu) 0693cff9 Use batch transactions when syncing (Howard Chu)
2017-01-15Check for correct thread before ending batch transactionHoward Chu1-0/+7
2017-01-14Don't cache block height, always get from DBHoward Chu2-14/+29
2017-01-14Use batch transactions when syncingHoward Chu5-10/+13
Faster throughput while avoiding corruption. I.e., makes running with --db-sync-mode safe more tolerable.
2017-01-13Merge pull request #1538Riccardo Spagni1-1/+0
aff28178 Remove db-auto-remove-logs (Miguel Herranz) 1229c685 Remove berkeley from db_type initialization (Miguel Herranz) e3090558 Show available types for db-type command (Miguel Herranz) 046ab33d Remove berkeley from blockchain_db_types (Miguel Herranz)
2017-01-10Remove berkeley from blockchain_db_typesMiguel Herranz1-1/+0
2017-01-07blockchain: allow marking "tx not found" without an exceptionmoneromooo-monero3-3/+27
This is a normal occurence in many cases, and there is no need to spam the log with those when it is.
2016-12-01db_lmdb: add info in an error message when we can't get an outputmoneromooo-monero1-1/+1
Will be useful to debug
2016-12-01db_lmdb: guard against going out of sync on unexpected db resultsmoneromooo-monero1-1/+1
m_num_outputs keeps track of the number of outputs, which should be the same as the size of both the output_txs and output_amounts databases. If one goes out of sync, we need to throw to abort whatever it is we were doing.
2016-12-01db_lmdb: minor pedantic tweaksmoneromooo-monero1-8/+8
Add consts in a few places where it makes sense, avoid unnecessary memory reallocation where we know the full size needed at the outset, simplify and avoid memory copy.
2016-12-01db_lmdb: set same packing format for output_data_t and pre_rct_output_data_tmoneromooo-monero1-0/+3
For safety, though it seems to have been the case already. Also add a comment about the necessary layout identity.
2016-10-15wallet: select part of the fake outs from recent outputsmoneromooo-monero3-12/+30
25% of the outputs are selected from the last 5 days (if possible), in order to avoid the common case of sending recently received outputs again. 25% and 5 days are subject to review later, since it's just a wallet level change.
2016-10-04Merge pull request #1139Riccardo Spagni1-2/+2
01ec195 Update CMakeLists.txt (codehalo) 446ebbc Update CMakeLists.txt (codehalo) bd773e7 Update CMakeLists.txt (codehalo) 3627cea Cleanup. Dropped "bit" from bitmonero. (Randi Joseph) 2a51396 Dropped "bit" from bitmonero. (Randi Joseph) 78b13d6 Cleanup. Dropped "bit" from bitmonero. (Randi Joseph) 1e6aedb Cleanup. Dropped "bit" from bitmonero. (Randi Joseph) 9e54616 Dropped "bit" from bitmonero. (Randi Joseph)
2016-09-29Merge pull request #1128Riccardo Spagni1-1/+1
d2e11f3 db_lmdb: do not error if dropping a non existent hard fork table (moneromooo-monero)
2016-09-26Dropped "bit" from bitmonero.Randi Joseph1-2/+2
2016-09-24db_lmdb: do not error if dropping a non existent hard fork tablemoneromooo-monero1-1/+1
2016-09-21Fix minor typo in "need resize" message.Myagui1-2/+2
Message observed while synchronizing a node from scratch. "LMDB memory map needs resized" Proposing a change to: "LMDB memory map needs to be resized"
2016-09-18cmake: transitive deps and remove deprecated LINK_*redfish1-8/+2
Keep the immediate direct deps at the library that depends on them, declare deps as PUBLIC so that targets that link against that library get the library's deps as transitive deps. Break dep cycle between blockchain_db <-> crytonote_core. No code refactoring, just hide cycle from cmake so that it doesn't complain (cycles are allowed only between static libs, not shared libs). This is in preparation for supproting BUILD_SHARED_LIBS cmake built-in option for building internal libs as shared.
2016-08-31core: faster find_blockchain_supplementmoneromooo-monero5-5/+16
Since this queries block heights for blocks that may or may not exist, queries for non existing blocks would throw an exception, and that would slow down the loop a lot. 7 seconds to go through a 30 hash list. Fix this by adding an optional return block height to block_exists and using this instead. Actual errors will still throw an exception. This also cuts down on log exception spam.
2016-08-29db_lmdb: fix reorg with "halfway rct" coinbase outputsmoneromooo-monero1-1/+3
2016-08-28New "Halfway RingCT" outputs for coinbase transactionsmoneromooo-monero1-2/+17
When RingCT is enabled, outputs from coinbase transactions are created as a single output, and stored as RingCT output, with a fake mask. Their amount is not hidden on the blockchain itself, but they are then able to be used as fake inputs in a RingCT ring. Since the output amounts are hidden, their "dustiness" is not an obstacle anymore to mixing, and this makes the coinbase transactions a lot smaller, as well as helping the TXO set to grow more slowly. Also add a new "Null" type of rct signature, which decreases the size required when no signatures are to be stored, as in a coinbase tx.
2016-08-28move the rct commitments to the output_amounts databasemoneromooo-monero6-167/+60
Since these are needed at the same time as the output pubkeys, this is a whole lot faster, and takes less space. Only outputs of 0 amount store the commitment. When reading other outputs, a fake commitment is regenerated on the fly. This avoids having to rewrite the database to add space for fake commitments for existing outputs. This code relies on two things: - LMDB must support fixed size records per key, rather than per database (ie, all records on key 0 are the same size, all records for non 0 keys are same size, but records from key 0 and non 0 keys do have different sizes). - the commitment must be directly after the rest of the data in outkey and output_data_t.
2016-08-28add rct to the protocolmoneromooo-monero2-0/+4
It is not yet constrained to a fork, so don't use on the real network or you'll be orphaned or rejected.
2016-08-28db_lmdb: update reset for recent db changesmoneromooo-monero1-0/+10
- we need to drop the new m_tx_indices database - we reset the version to current version This fixes the core tests failing to initialize.
2016-08-28blockchain_db: add functions for adding/removing/getting rct commitmentsmoneromooo-monero3-0/+153
2016-08-26db_lmdb: do not try to modify the database in read only modemoneromooo-monero1-4/+12
2016-08-11Merge pull request #948Riccardo Spagni3-9/+28
11dc091 Fake outs set is now decided by the wallet (moneromooo-monero) 1593553 new unlocked parameter to output_histogram (moneromooo-monero)
2016-08-10db_lmdb: errors dropping hf starting height db on reset are not fatalmoneromooo-monero1-2/+1
This db is now dropped unconditionally, so may or may not be there in the first place.
2016-08-01new unlocked parameter to output_histogrammoneromooo-monero3-9/+28
This constrains the number of instances of any amount to the unlocked ones (as defined by the default unlock time setting: outputs with non default unlock time are not considered, so may be counted as unlocked even if they are not actually unlocked).
2016-07-26db_lmdb: add some missing api call checksmoneromooo-monero1-14/+31
2016-07-13remove hf_starting_height dbmoneromooo-monero5-127/+2
It's not really needed, it used to be an optimization for when that code was not using the db and needed to recalculate things fast on startup.
2016-04-29Merge branch 'performance' of https://github.com/LMDB/bitmoneroRiccardo Spagni4-761/+1210
2016-04-28add a --max-concurrency flagmoneromooo-monero1-1/+1
It sets the max number of threads to use for a parallel job. This is different that the number of total threads, since monero binaries typically start a lot of them.
2016-04-24Another take on migrationHoward Chu2-582/+106
Delete old indices and recreate them, rather than updating them Maybe not quite as slow as before.
2016-04-08MigrationHoward Chu2-5/+1049
Migrate from DB version 0 to version 1 on startup
2016-04-05Merge branch 'performance' into masterHoward Chu4-747/+628
2016-04-05CleanupHoward Chu3-24/+65
drop obsolete remove_output() fix get_output_key(global), fix crash in blockchain_dump
2016-04-05Cleanup and clarifyHoward Chu4-146/+129
Try to rationalize the variable names, document usage.
2016-04-05Use cursors in some remove functionsHoward Chu1-7/+33
Helps when they're called repeatedly in one txn
2016-04-05Use DUPFIXED for block_info and output_txsHoward Chu2-56/+46
Saves another ~150MB or so on the full blockchain
2016-04-05More outputs consolidationHoward Chu2-491/+263
Also bumped DB VERSION to 1 Another significant speedup and space savings: Get rid of global_output_indices, remove indirection from output to keys This is the change warptangent described on irc but never got to finish.
2016-04-05Use DUPFIXED for output_keysHoward Chu1-23/+37
Saves another 90MB on 200000 block import. Had to bring back compare_uint64 for this, but it's safe since this table is always 64-bit aligned.
2016-04-05Use DUPFIXED for tx_indicesHoward Chu1-50/+60
Small space savings, no measurable speedup
2016-04-05Use DUPFIXED for block_heightsHoward Chu1-22/+29
Only a small savings...
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-27 21:08:54 +0000