| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
There will need to be some more refactoring for these changes to be
considered complete/correct, but for now it's working.
new daemon cli argument "--db-type", works for LMDB and BerkeleyDB.
A good deal of refactoring is also present in this commit, namely
Blockchain no longer instantiates BlockchainDB, but rather is passed a
pointer to an already-instantiated BlockchainDB on init().
|
|
|
|
|
|
|
|
51e3579 Fixed bug in static linking boost on MINGW (Thomas Winget)
f78bb00 Hopefully fixes build on Windows for real this time (Thomas Winget)
2b0583b Hopefully fixes build on Windows (Thomas Winget)
9dab105 DNS checkpoint loading for testnet should now be correct (Thomas Winget)
52f9629 sending commands to forked daemon works on testnet now (Thomas Winget)
76289d0 Fix tests building -- function signatures changed (Thomas Winget)
db53e19 revert stop_daemon method to use correct exit (Thomas Winget)
96cbecf RPC calls for background daemon added in (Thomas Winget)
9193d6f Daemonize changes pulled in -- daemon builds (Thomas Winget)
|
|
|
|
Preparation for PR
|
|
Add option "--resume <on|off>" where default is on.
|
|
|
|
|
|
Where this method is used, a BlockchainDB object is always expected, so
a pointer is unnecessary and less safe.
|
|
|
|
There was a workaround for linking to boost at all on MINGW, but
unfortunately this workaround would not correctly link to boost
statically. This workaround for that workaround works around the issue
that that workaround had.
|
|
|
|
|
|
|
|
|
|
|
|
Because of some github fuckery I had to manually cherry-pick/rebase
fluffypony's 32- and 64-bit lmdb changes, hence this isn't actually a
merge commit.
|
|
Also change LMDB Cmake variables to CACHE rather than upgrading them
through several parent scopes.
|
|
|
|
|
|
CMakeList as that is not part of standard Unbound
|
|
|
|
|
|
|
|
This reverts commit b21335642e75b35d3b178a754f4cdb2314989cd1.
|
|
This reverts commit 8b82f3c57ffdc4dc190f4755c61d74d8359c8c0f.
|
|
|
|
This allows an LMDB database to be used as the blockchain to export.
Adjust SOURCE_DB in src/blockchain_converter/blockchain_export.h
depending on needs. Defaults to DB_MEMORY.
DB_MEMORY is a sensible default for users migrating to LMDB, as it
allows the exporter to use the in-memory blockchain while the other
binaries work with LMDB, without recompiling anything.
|
|
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working. Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.
Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
so block numbers *in the database* will be 1-indexed. Modifications
to indexing have been made as needed.
|
|
Make Cmake things aware of BerkeleyDB and BlockchainBDB
Make the BlockchainDB unit tests aware of BlockchainBDB
|
|
LMDB implementation code copy/paste/modified into the Berkeley DB
implementation. Need to test if it builds, then if it works, and so on,
but the code is all there.
|
|
|
|
Basically verbatim copy of LMDB implementation, but with the guts ripped
out and includes changed, etc.
|
|
Based on work by tomerkon.
See https://github.com/tomerkon/bitmonero
src/cryptonote_core/bootfilesaver.{h,cpp}
src/bootfilegen/bootfilegen.cpp
|
|
Add support to:
- BlockchainDB, BlockchainLMDB
- blockchain_import utility to open LMDB database with one or more
LMDB flags.
Sample use:
$ blockchain_import --database lmdb#nosync
$ blockchain_import --database lmdb#nosync,nometasync
|
|
|
|
This imports to the blockchain database from an exported blockchain
file.
It can be used to bootstrap a new database or to add blocks to an
existing one.
Supports:
- both the in-memory and LMDB implementations
- optional: batching, verification, testnet
See help for usage.
Based on work by tomerkon.
See https://github.com/tomerkon
src/cryptonote_core/bootfileloader.{h,cpp}
|
|
This handling may be changed in the future.
|
|
Add log level support.
Add testnet support.
Add command-line options:
--help
--data-dir
--testnet-data-dir
--testnet
--log-level
--batch
--batch-size
--block-number
See help for usage. Run at log level 1 to see profiling stats.
|
|
|
|
In order to make things more general, BlockchainDB now has get_db_name()
which should return a string with the "name" of that type of db.
This "name" will be the subfolder name that holds that db type's files
within the monero folder.
Small bugfix: blockchain_converter was not correctly appending this in
the prior hard-coded-string implementation of the subfolder data
directory concept.
|
|
|
|
Forgot that CMake vars set to PARENT_SCOPE will still vanish if that
parent scope goes...out of scope. LMDB vars elevated one more scope to
compensate for moving db_drivers/ into external/
|
|
7e72e94 update checkpoint (smooth)
|
|
|
|
Ostensibly janitorial work, but should be more relevant later down the
line. Things that depend on core cryptonote things (i.e.
cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
|
|
|
|
41a95e7 add comment about avoiding overflow (smooth)
754a785 minimum subsidy for mining incentives, remove unused LEGACY_FEE define (smooth)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7590f33 move website and DNS unit tests from monero.cc to getmonero.org (Riccardo Spagni)
|
|
|
|
This was changed because sometimes the daemon does not complete its exit
routine with this method, but as it does correctly wind most things down
even if it gets stuck I've changed it back.
|
|
The RPC calls the daemon executable uses to talk to the running daemon
instance have mostly been added back in. Rate limiting has not been
added in upstream, but is on its way in a separate effort, so those
calls are still NOPed out.
|
|
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return). Other than that, this *should*
work...at any rate, it builds, and that's something.
|
|
|
|
Add support to BlockchainDB and BlockchainLMDB for batch transactions.
Add profiling to block and tx processing and DB operations.
Improve block and tx processing efficiency by less repeat hashing.
Move LMDB storage to "lmdb" subfolder.
- Upon startup, if old LMDB files are detected, abort with a message for the user to move them to subfolder or delete them.
Update and fix log statements and formatting.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
BlockchainLMDB::add_block()
BlockchainLMDB::add_transaction_data()
BlockchainDB::add_transaction()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fix Blockchain::get_tail_id() to set parameter to last block number instead of height
|
|
of height
This reflects the behavior of blockchain_storage::get_tail_id().
Fixes #27 so that RPC method getlastblockheader works.
|
|
This reverts commit 6f1c4b4c2c78c930fe30ed648e855a6ce55f7dcd.
|
|
|
|
7b01a07 added tx size to incoming_transfers RPC Call (Riccardo Spagni)
|
|
|
|
|
|
3ece158 make fallback compatible with newer slow-hash (Riccardo Spagni)
709bbc5 replaced 64-bit multiplication in difficulty.cpp with a portable version (Riccardo Spagni)
497a514 replaced 64-bit multiplication in difficulty.cpp with a portable version (Riccardo Spagni)
|
|
|
|
|
|
Usage:
default is lmdb for blockchain branch:
$ make release
same as:
$ DATABASE=lmdb make release
for original in-memory implementation:
$ DATABASE=memory make release
COMMITS:
- Add compile-time support for both db implementations: in-memory and LMDB
NOTE:
The default should be changed to lmdb when this is merged upstream
unless we're 100% sure the LMDB implementation is ready.
|
|
Usage:
default is lmdb for blockchain branch:
$ make release
same as:
$ DATABASE=lmdb make release
for original in-memory implementation:
$ DATABASE=memory make release
|
|
|
|
See commit 4ba680f2946966df2030e5765e40ee0a36b112c4
|
|
See commit cf5a8b1d6c3df615641e81328bb3d8cf80cd70e3
|
|
Update to match LOG_PRINT_RED_Lx statements.
See commit cf5a8b1d6c3df615641e81328bb3d8cf80cd70e3
|
|
get_tx_outputs_gindexs()
|
|
This would have been triggered if function was called without fourth
parameter and ring signature check failed.
|
|
|
|
Merge branch 'monero-project/master' into blockchain
|
|
|
|
|
|
|
|
|
|
9b3673a added portable slow-hash, updated makefile targets, fixed readme (Riccardo Spagni)
|
|
|
|
e70bc3d fixed sse2 ifdef (Riccardo Spagni)
|
|
|
|
2b3b70a remove aes flag, detect no sse2 (Riccardo Spagni)
|
|
|
|
|
|
|
|
Merge 'monero-project/master' with blockchain
|
|
|
|
Working on Linux, needs verified for other platforms but should be fine.
|
|
|
|
24d500c Add a --restricted-rpc flag to simplewallet (moneromooo-monero)
|
|
87839cd Allow get_bulk_payments to return all payments regardless of payment ID (moneromooo-monero)
|
|
Fix seed node threaded DNS lookup
|
|
Remove code previously made unused and marked unused
|
|
Use copied value of seed node index during thread creation, not
reference.
- fixes segfault
Use boost::thread::try_join_until() instead of an atomic flag result
variable for each thread.
Add and handle interrupt for thread timeout.
- fixes segfault where a thread exceeds requested timeout and tries to
assign results to a referenced, but now out-of-scope, variable in
the main thread.
|
|
|
|
Fixes and additions for block reorganization
|
|
It expects the total number of blocks of main chain, not last block id
(off-by-one error).
This again behaves like the same height assertion done in original
implementation in blockchain_storage::handle_alternative_block().
This allows a reorganization to proceed after an alternative block has
been added.
|
|
difficulty.
This fixes the continual reorganization between a main and alternate
chain, using the same two latest blocks from each.
The check that cumulative difficulty of the alternate chain is bigger
than main's was not using main's last block, but incorrectly using the
passed-in block's previous block. main_chain_cumulative_difficulty was
being used in two different ways. This has been split up to keep use
of main_chain_cumulative_difficulty consistent.
|
|
Remove have_block() check from Blockchain::handle_block_to_main_chain().
Add logging to have_block().
This allows blockchain reorganization to proceed further.
have_block() check here causes an error after a blockchain reorganize
begins with error: "Attempting to add block to main chain, but it's
already either there or in an alternate chain."
While reorganizing to become the main chain, a block in the
alternative chain would be refused due to have_block() rightfully
finding it in the alternative chain. The reorganization would end in
rollback, restoring to previous blockchain.
Original implementation didn't call it here, and it doesn't appear
necessary to be called from here in this implementation either. When
needed, it appears it's called prior to handle_block_to_main_chain().
|
|
Complete method BlockchainLMDB::remove_output()
- use output index as the key for:
m_output_indices, m_output_txs, m_output_keys
- call new method BlockchainLMDB::remove_amount_output_index()
Add method to remove amount output index.
- BlockchainLMDB::remove_amount_output_index()
- for m_output_amounts
This also fixes the segfault when blockchain reorganization is
attempted.
|
|
Use last block id, not number of blocks (off-by-one error).
Fixes error at start of blockchain reorganization: "Attempt to get
cumulative difficulty from height <XXXXXX> failed -- difficulty not in
db"
|
|
|
|
It restricts RPC to a subset of "view only" commands. Kind of like
a poor man's view key replacement.
|
|
by giving an empty list of payment IDs.
|
|
Fix transfers
Output indexing was being handled improperly. These changes fix that. Wallets (re-)created using this branch will need to be (re-)re-created in order to get correct output indices.
|
|
Implement BlockchainLMDB::get_output_global_index()
- returns global output index for a given amount and amount output
index.
Add information to debug statement for failed ring signature check
within Blockchain::check_tx_inputs()
Fixes bitmonerod RPC call "/getrandom_outs.bin" to return correct
output keys, used in creating a transaction with mixins.
TODO: get_output_global_index() could be refactored with part of
get_output_tx_and_index() as the latter uses the former's
functionality. Keep track of LMDB read transaction.
|
|
Fix Blockchain::get_tx_outputs_gindexs() to return amount output
indices.
Implement BlockchainLMDB::get_tx_amount_output_indices() and call it
from the function instead of BlockchainLMDB::get_tx_output_indices()
Previously, Blockchain::get_tx_outputs_gindexs() was instead returning
global output indices, which are internal to LMDB databases.
Allows bitmonerod RPC /get_o_indexes.bin to return the amount output
indices as expected.
Allows simplewallet refresh to set correct amount output indices for
incoming transfers. simplewallet can now construct and send valid
transactions (currently only without mixins).
This is a fix that doesn't require altering the structure of the
current LMDB databases.
TODO:
This can be done more efficiently by adding another LMDB database
(key-value table).
It's not used during regular transaction validation by bitmonerod. I
think it's currently used only or mainly by simplewallet for just its
own incoming transactions. So the current behavior is not a primary
bottleneck.
Currently, it's using the "output_amounts" database, walking through a
given amount's list of values, comparing each one to a given global
output index. The iteration number of the match is the desired result:
the amount output index. This is done for each global output index of
the transaction.
A tx's amount output indices can be stored in various other ways
allowing for faster lookup. Since a tx is only written once, there are
no special future write requirements for its list of indices.
|
|
As it is useful for functions calling BlockchainDB functions to know
whether an exception is expected (attempting to get a block that doesn't
exist and counting it missing if not, to save time checking if it
does, for example), the inline functions throw{0,1} need to keep the
exception type information.
Slight comment update due to copy/paste failure.
|
|
This commit should fix the segfault in
Blockchain::handle_alternative_block, and also updates a few comments
that were either incorrect or incomplete.
|
|
1b46226 std::atomic_flag has no copy/move constructor, can't have a vector (Thomas Winget)
df53c0a small typo in previous commit (Thomas Winget)
4a53898 DNS seed timeout and fallback (Thomas Winget)
|
|
|
|
7de1a2d previous hash added to GBT (Riccardo Spagni)
|
|
|
|
Obtain tx hash and tx output index from amount and output offset
|
|
Blockchain
|
|
Fixes problem of obtaining incorrect outputs used for tx input.
Reverts to earlier intended behavior that was fixed in previous
commit's split of get_output_tx_and_index into two functions.
|
|
Thanks to moneromooo-monero for spotting the bug.
|
|
This finally fixes raw tx being accepted by the daemon.
|
|
|
|
|
|
db_lmdb: catch attempt to remove block from an empty blockchain
db_lmdb: do not give the group database write permissions
|
|
|
|
|
|
It would probably have thrown when not finding a block at
height 2^64-1, but better make things clear.
|
|
db_lmdb: factor all the log+throw code paths
|
|
many const-correctness changes and code quality enhancements. minor bugfix on block removal.
|
|
|
|
Blockchain
lots of const-correctness and more proper code, bug fix with syncing.
|
|
It makes the code simpler, avoids possible copy/paste errors
(wrong sizeof, etc), and generally unclutters the calling code.
|
|
When the wallet syncs from the first block, it is fine to start
at the genesis block.
|
|
|
|
|
|
It is already there (unless overridden via command line).
|
|
|
|
|
|
Add lmdb to the cmake gunk, and use actual user directory for the blockchain location
|
|
|
|
|
|
The existing assert is kept as it is stricter than the function's
internal assert.
|
|
|
|
While the dtor implementation does not actually do anything, other
paths do delete it, and the dtor might do someting later.
|
|
|
|
|
|
|
|
Ideally, the log would go in the exception's ctor, but
two log levels are used, so I'd need to specify the level
in the ctor, which isn't great as it's not really related
to the exception.
|
|
credit here:
https://bitcointalk.org/index.php?topic=583449.msg9562845#msg9562845
|
|
|
|
|
|
update for rebase (warptangent 2015-01-04)
src/cryptonote_core/CMakeLists.txt (edit)
- replace LMDB_LIBRARIES with LMDB_LIBRARY set from autodetection
|
|
|
|
|
|
hard-coded config folder, hard-coded BlockchainDB subclass.
Needs finessing, but should be testable this way.
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
src/CMakeLists.txt (edit original commit)
src/blockchain_converter/CMakeLists.txt (add)
|
|
|
|
|
|
|
|
Minor bugfixes in block removal
Storing outputs outside their transactions is largely unnecessary, and
thus has been removed.
|
|
|
|
* - Well, mostly. Haven't let it sync too far just yet. Currently
trying to figure out the best way to deal with LMDB/mmap virtual memory
pages.
|
|
There are quite a few debug prints in this commit that will need removed
later, but for posterity (in case someone wants to debug this while I'm
away), I left them in.
Currently errors when syncing on the first block that has a "real"
transaction. Seems to not be able to validate the ring signature, but I
can't for the life of me figure out what's going wrong.
|
|
Blockchain and BlockchainLMDB classes now have a debug print at the
beginning of each function at log level 2. These can be removed at any
time, but for now are quite useful.
Blockchain runs, and adds the genesis block just fine, but for some
reason isn't getting new blocks.
|
|
|
|
Probably needs more looking at -- lot of things were done...in a rushed
sort of way. That said, it all builds and *should* be at least
testable.
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
src/CMakeLists.txt (remove edits from original commit)
tests/CMakeLists.txt (remove edits from original commit)
src/cryptonote_core/CMakeLists.txt (edit)
- use blockchain db .cpp and .h files
- add LMDB_LIBRARIES
|
|
All of the functionality for the LMDB implementation of BlockchainDB is
implemented, but only what is in tests/unit_tests/BlockchainDB.cpp has
been tested. This is basically add a block, see if you can get the
block and a tx from the block. More tests should be added at some
point.
|
|
|
|
Still needs testing (and need to write a few more unit tests), but
everything should be there. Lots of unfortunate duplication,
but...well, I can't see a way around it using LMDB.
A couple of other minor changes in this commit, only slightly relevant.
|
|
The rest should just be tedious copypasta and modification.
|
|
Some BlockchainDB unit testing fleshed out (and working), rudimentary
linker flag for lmdb in CMakeLists, but should probably be done
"correctly" at some point (find it on whatever system you're building on
and all that jazz).
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
tests/CMakeLists.txt (remove edits from original commit)
|
|
|
|
miscellaneous changes to BlockchainDB/blockchain as well, namely
replacing instances of std::list with std::vector
|
|
libglim is an Apache-licensed C++ wrapper for lmdb, and rather than
rolling our own it seems prudent to use it.
Note: lmdb is not included in it, and unless something happens as did
with libunbound, should be installed via each OS' package manager or
equivalent.
|
|
Still need to add in the new checkpointing functionality, as well as
touch up a few things, but is okay for now.
|
|
|
