| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This reverts commit b21335642e75b35d3b178a754f4cdb2314989cd1.
|
|
This reverts commit 8b82f3c57ffdc4dc190f4755c61d74d8359c8c0f.
|
|
|
|
This allows an LMDB database to be used as the blockchain to export.
Adjust SOURCE_DB in src/blockchain_converter/blockchain_export.h
depending on needs. Defaults to DB_MEMORY.
DB_MEMORY is a sensible default for users migrating to LMDB, as it
allows the exporter to use the in-memory blockchain while the other
binaries work with LMDB, without recompiling anything.
|
|
Everything except actually *using* BlockchainBDB is wired up, but the db
itself is not yet working. Some error about user mem not large enough.
I think I know what this error means, but I can't determine the cause.
Notes: BerkeleyDB does not allow 0-indexing in its recno type databases,
so block numbers *in the database* will be 1-indexed. Modifications
to indexing have been made as needed.
|
|
Make Cmake things aware of BerkeleyDB and BlockchainBDB
Make the BlockchainDB unit tests aware of BlockchainBDB
|
|
LMDB implementation code copy/paste/modified into the Berkeley DB
implementation. Need to test if it builds, then if it works, and so on,
but the code is all there.
|
|
|
|
Basically verbatim copy of LMDB implementation, but with the guts ripped
out and includes changed, etc.
|
|
Based on work by tomerkon.
See https://github.com/tomerkon/bitmonero
src/cryptonote_core/bootfilesaver.{h,cpp}
src/bootfilegen/bootfilegen.cpp
|
|
Add support to:
- BlockchainDB, BlockchainLMDB
- blockchain_import utility to open LMDB database with one or more
LMDB flags.
Sample use:
$ blockchain_import --database lmdb#nosync
$ blockchain_import --database lmdb#nosync,nometasync
|
|
|
|
This imports to the blockchain database from an exported blockchain
file.
It can be used to bootstrap a new database or to add blocks to an
existing one.
Supports:
- both the in-memory and LMDB implementations
- optional: batching, verification, testnet
See help for usage.
Based on work by tomerkon.
See https://github.com/tomerkon
src/cryptonote_core/bootfileloader.{h,cpp}
|
|
This handling may be changed in the future.
|
|
Add log level support.
Add testnet support.
Add command-line options:
--help
--data-dir
--testnet-data-dir
--testnet
--log-level
--batch
--batch-size
--block-number
See help for usage. Run at log level 1 to see profiling stats.
|
|
|
|
|
|
|
|
In order to make things more general, BlockchainDB now has get_db_name()
which should return a string with the "name" of that type of db.
This "name" will be the subfolder name that holds that db type's files
within the monero folder.
Small bugfix: blockchain_converter was not correctly appending this in
the prior hard-coded-string implementation of the subfolder data
directory concept.
|
|
|
|
Forgot that CMake vars set to PARENT_SCOPE will still vanish if that
parent scope goes...out of scope. LMDB vars elevated one more scope to
compensate for moving db_drivers/ into external/
|
|
|
|
7e72e94 update checkpoint (smooth)
|
|
|
|
Ostensibly janitorial work, but should be more relevant later down the
line. Things that depend on core cryptonote things (i.e.
cryptonote_core) don't necessarily depend on BlockchainDB and thus
have no need to have BlockchainDB baked in with them.
|
|
|
|
41a95e7 add comment about avoiding overflow (smooth)
754a785 minimum subsidy for mining incentives, remove unused LEGACY_FEE define (smooth)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7590f33 move website and DNS unit tests from monero.cc to getmonero.org (Riccardo Spagni)
|
|
|
|
|
|
+toc -doc -drmonero
Fixed the windows path, and improved logging and data
(for graph) logging, fixed some locks and added more checks.
Still there is a locking error,
not added by my patches, but present in master version
(locking of map/list of peers).
|
|
This was changed because sometimes the daemon does not complete its exit
routine with this method, but as it does correctly wind most things down
even if it gets stuck I've changed it back.
|
|
The RPC calls the daemon executable uses to talk to the running daemon
instance have mostly been added back in. Rate limiting has not been
added in upstream, but is on its way in a separate effort, so those
calls are still NOPed out.
|
|
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return). Other than that, this *should*
work...at any rate, it builds, and that's something.
|
|
|
|
Add support to BlockchainDB and BlockchainLMDB for batch transactions.
Add profiling to block and tx processing and DB operations.
Improve block and tx processing efficiency by less repeat hashing.
Move LMDB storage to "lmdb" subfolder.
- Upon startup, if old LMDB files are detected, abort with a message for the user to move them to subfolder or delete them.
Update and fix log statements and formatting.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
BlockchainLMDB::add_block()
BlockchainLMDB::add_transaction_data()
BlockchainDB::add_transaction()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fix Blockchain::get_tail_id() to set parameter to last block number instead of height
|
|
of height
This reflects the behavior of blockchain_storage::get_tail_id().
Fixes #27 so that RPC method getlastblockheader works.
|
|
This reverts commit 6f1c4b4c2c78c930fe30ed648e855a6ce55f7dcd.
|
|
|
|
new update of the pr with network limits
more debug options:
discarding downloaded blocks all or after given height.
trying to trigger the locking errors.
debug levels polished/tuned to sane values.
debug/logging improved.
warning: this pr should be correct code, but it could make
an existing (in master version) locking error appear more often.
it's a race on the list (map) of peers, e.g. between closing/deleting
them versus working on them in net-limit sleep in sending chunk.
the bug is not in this code/this pr, but in the master version.
the locking problem of master will be fixed in other pr.
problem is ub, and in practice is seems to usually cause program abort
(tested on debian stable with updated gcc). see --help for option
to add sleep to trigger the error faster.
|
|
thought it was already fixed, apparently commit got lost somewhere
|
|
|
|
Update of the PR with network limits
works very well for all speeds
(but remember that low download speed can stop upload
because we then slow down downloading of blockchain
requests too)
more debug options
fixed pedantic warnings in our code
should work again on Mac OS X and FreeBSD
fixed warning about size_t
tested on Debian, Ubuntu, Windows(testing now)
TCP options and ToS (QoS) flag
FIXED peer number limit
FIXED some spikes in ingress/download
FIXED problems when other up and down limit
|
|
commands and options for network limiting
works very well e.g. for 50 KiB/sec up and down
ToS (QoS) flag
peer number limit
TODO some spikes in ingress/download
TODO problems when other up and down limit
added "otshell utils" - simple logging (with colors, text files channels)
|
|
7b01a07 added tx size to incoming_transfers RPC Call (Riccardo Spagni)
|
|
|
|
|
|
3ece158 make fallback compatible with newer slow-hash (Riccardo Spagni)
709bbc5 replaced 64-bit multiplication in difficulty.cpp with a portable version (Riccardo Spagni)
497a514 replaced 64-bit multiplication in difficulty.cpp with a portable version (Riccardo Spagni)
|
|
|
|
|
|
Usage:
default is lmdb for blockchain branch:
$ make release
same as:
$ DATABASE=lmdb make release
for original in-memory implementation:
$ DATABASE=memory make release
COMMITS:
- Add compile-time support for both db implementations: in-memory and LMDB
NOTE:
The default should be changed to lmdb when this is merged upstream
unless we're 100% sure the LMDB implementation is ready.
|
|
Usage:
default is lmdb for blockchain branch:
$ make release
same as:
$ DATABASE=lmdb make release
for original in-memory implementation:
$ DATABASE=memory make release
|
|
|
|
See commit 4ba680f2946966df2030e5765e40ee0a36b112c4
|
|
See commit cf5a8b1d6c3df615641e81328bb3d8cf80cd70e3
|
|
Update to match LOG_PRINT_RED_Lx statements.
See commit cf5a8b1d6c3df615641e81328bb3d8cf80cd70e3
|
|
get_tx_outputs_gindexs()
|
|
This would have been triggered if function was called without fourth
parameter and ring signature check failed.
|
|
|
|
Merge branch 'monero-project/master' into blockchain
|
|
|
|
|
|
|
|
|
|
9b3673a added portable slow-hash, updated makefile targets, fixed readme (Riccardo Spagni)
|
|
|
|
e70bc3d fixed sse2 ifdef (Riccardo Spagni)
|
|
|
|
2b3b70a remove aes flag, detect no sse2 (Riccardo Spagni)
|
|
|
|
|
|
|
|
Merge 'monero-project/master' with blockchain
|
|
|
|
Working on Linux, needs verified for other platforms but should be fine.
|
|
|
|
24d500c Add a --restricted-rpc flag to simplewallet (moneromooo-monero)
|
|
87839cd Allow get_bulk_payments to return all payments regardless of payment ID (moneromooo-monero)
|
|
Fix seed node threaded DNS lookup
|
|
Remove code previously made unused and marked unused
|
|
Use copied value of seed node index during thread creation, not
reference.
- fixes segfault
Use boost::thread::try_join_until() instead of an atomic flag result
variable for each thread.
Add and handle interrupt for thread timeout.
- fixes segfault where a thread exceeds requested timeout and tries to
assign results to a referenced, but now out-of-scope, variable in
the main thread.
|
|
|
|
Fixes and additions for block reorganization
|
|
It expects the total number of blocks of main chain, not last block id
(off-by-one error).
This again behaves like the same height assertion done in original
implementation in blockchain_storage::handle_alternative_block().
This allows a reorganization to proceed after an alternative block has
been added.
|
|
difficulty.
This fixes the continual reorganization between a main and alternate
chain, using the same two latest blocks from each.
The check that cumulative difficulty of the alternate chain is bigger
than main's was not using main's last block, but incorrectly using the
passed-in block's previous block. main_chain_cumulative_difficulty was
being used in two different ways. This has been split up to keep use
of main_chain_cumulative_difficulty consistent.
|
|
Remove have_block() check from Blockchain::handle_block_to_main_chain().
Add logging to have_block().
This allows blockchain reorganization to proceed further.
have_block() check here causes an error after a blockchain reorganize
begins with error: "Attempting to add block to main chain, but it's
already either there or in an alternate chain."
While reorganizing to become the main chain, a block in the
alternative chain would be refused due to have_block() rightfully
finding it in the alternative chain. The reorganization would end in
rollback, restoring to previous blockchain.
Original implementation didn't call it here, and it doesn't appear
necessary to be called from here in this implementation either. When
needed, it appears it's called prior to handle_block_to_main_chain().
|
|
Complete method BlockchainLMDB::remove_output()
- use output index as the key for:
m_output_indices, m_output_txs, m_output_keys
- call new method BlockchainLMDB::remove_amount_output_index()
Add method to remove amount output index.
- BlockchainLMDB::remove_amount_output_index()
- for m_output_amounts
This also fixes the segfault when blockchain reorganization is
attempted.
|
|
Use last block id, not number of blocks (off-by-one error).
Fixes error at start of blockchain reorganization: "Attempt to get
cumulative difficulty from height <XXXXXX> failed -- difficulty not in
db"
|
|
|
|
It restricts RPC to a subset of "view only" commands. Kind of like
a poor man's view key replacement.
|
|
by giving an empty list of payment IDs.
|
|
Fix transfers
Output indexing was being handled improperly. These changes fix that. Wallets (re-)created using this branch will need to be (re-)re-created in order to get correct output indices.
|
|
Implement BlockchainLMDB::get_output_global_index()
- returns global output index for a given amount and amount output
index.
Add information to debug statement for failed ring signature check
within Blockchain::check_tx_inputs()
Fixes bitmonerod RPC call "/getrandom_outs.bin" to return correct
output keys, used in creating a transaction with mixins.
TODO: get_output_global_index() could be refactored with part of
get_output_tx_and_index() as the latter uses the former's
functionality. Keep track of LMDB read transaction.
|
|
Fix Blockchain::get_tx_outputs_gindexs() to return amount output
indices.
Implement BlockchainLMDB::get_tx_amount_output_indices() and call it
from the function instead of BlockchainLMDB::get_tx_output_indices()
Previously, Blockchain::get_tx_outputs_gindexs() was instead returning
global output indices, which are internal to LMDB databases.
Allows bitmonerod RPC /get_o_indexes.bin to return the amount output
indices as expected.
Allows simplewallet refresh to set correct amount output indices for
incoming transfers. simplewallet can now construct and send valid
transactions (currently only without mixins).
This is a fix that doesn't require altering the structure of the
current LMDB databases.
TODO:
This can be done more efficiently by adding another LMDB database
(key-value table).
It's not used during regular transaction validation by bitmonerod. I
think it's currently used only or mainly by simplewallet for just its
own incoming transactions. So the current behavior is not a primary
bottleneck.
Currently, it's using the "output_amounts" database, walking through a
given amount's list of values, comparing each one to a given global
output index. The iteration number of the match is the desired result:
the amount output index. This is done for each global output index of
the transaction.
A tx's amount output indices can be stored in various other ways
allowing for faster lookup. Since a tx is only written once, there are
no special future write requirements for its list of indices.
|
|
As it is useful for functions calling BlockchainDB functions to know
whether an exception is expected (attempting to get a block that doesn't
exist and counting it missing if not, to save time checking if it
does, for example), the inline functions throw{0,1} need to keep the
exception type information.
Slight comment update due to copy/paste failure.
|
|
This commit should fix the segfault in
Blockchain::handle_alternative_block, and also updates a few comments
that were either incorrect or incomplete.
|
|
1b46226 std::atomic_flag has no copy/move constructor, can't have a vector (Thomas Winget)
df53c0a small typo in previous commit (Thomas Winget)
4a53898 DNS seed timeout and fallback (Thomas Winget)
|
|
|
|
7de1a2d previous hash added to GBT (Riccardo Spagni)
|
|
|
|
Obtain tx hash and tx output index from amount and output offset
|
|
Blockchain
|
|
Fixes problem of obtaining incorrect outputs used for tx input.
Reverts to earlier intended behavior that was fixed in previous
commit's split of get_output_tx_and_index into two functions.
|
|
Thanks to moneromooo-monero for spotting the bug.
|
|
This finally fixes raw tx being accepted by the daemon.
|
|
|
|
|
|
db_lmdb: catch attempt to remove block from an empty blockchain
db_lmdb: do not give the group database write permissions
|
|
|
|
|
|
It would probably have thrown when not finding a block at
height 2^64-1, but better make things clear.
|
|
db_lmdb: factor all the log+throw code paths
|
|
many const-correctness changes and code quality enhancements. minor bugfix on block removal.
|
|
|
|
Blockchain
lots of const-correctness and more proper code, bug fix with syncing.
|
|
It makes the code simpler, avoids possible copy/paste errors
(wrong sizeof, etc), and generally unclutters the calling code.
|
|
When the wallet syncs from the first block, it is fine to start
at the genesis block.
|
|
|
|
|
|
It is already there (unless overridden via command line).
|
|
|
|
|
|
Add lmdb to the cmake gunk, and use actual user directory for the blockchain location
|
|
|
|
|
|
The existing assert is kept as it is stricter than the function's
internal assert.
|
|
|
|
While the dtor implementation does not actually do anything, other
paths do delete it, and the dtor might do someting later.
|
|
|
|
|
|
|
|
Ideally, the log would go in the exception's ctor, but
two log levels are used, so I'd need to specify the level
in the ctor, which isn't great as it's not really related
to the exception.
|
|
credit here:
https://bitcointalk.org/index.php?topic=583449.msg9562845#msg9562845
|
|
|
|
|
|
update for rebase (warptangent 2015-01-04)
src/cryptonote_core/CMakeLists.txt (edit)
- replace LMDB_LIBRARIES with LMDB_LIBRARY set from autodetection
|
|
|
|
|
|
hard-coded config folder, hard-coded BlockchainDB subclass.
Needs finessing, but should be testable this way.
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
src/CMakeLists.txt (edit original commit)
src/blockchain_converter/CMakeLists.txt (add)
|
|
|
|
|
|
|
|
Minor bugfixes in block removal
Storing outputs outside their transactions is largely unnecessary, and
thus has been removed.
|
|
|
|
* - Well, mostly. Haven't let it sync too far just yet. Currently
trying to figure out the best way to deal with LMDB/mmap virtual memory
pages.
|
|
There are quite a few debug prints in this commit that will need removed
later, but for posterity (in case someone wants to debug this while I'm
away), I left them in.
Currently errors when syncing on the first block that has a "real"
transaction. Seems to not be able to validate the ring signature, but I
can't for the life of me figure out what's going wrong.
|
|
Blockchain and BlockchainLMDB classes now have a debug print at the
beginning of each function at log level 2. These can be removed at any
time, but for now are quite useful.
Blockchain runs, and adds the genesis block just fine, but for some
reason isn't getting new blocks.
|
|
|
|
Probably needs more looking at -- lot of things were done...in a rushed
sort of way. That said, it all builds and *should* be at least
testable.
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
src/CMakeLists.txt (remove edits from original commit)
tests/CMakeLists.txt (remove edits from original commit)
src/cryptonote_core/CMakeLists.txt (edit)
- use blockchain db .cpp and .h files
- add LMDB_LIBRARIES
|
|
All of the functionality for the LMDB implementation of BlockchainDB is
implemented, but only what is in tests/unit_tests/BlockchainDB.cpp has
been tested. This is basically add a block, see if you can get the
block and a tx from the block. More tests should be added at some
point.
|
|
|
|
Still needs testing (and need to write a few more unit tests), but
everything should be there. Lots of unfortunate duplication,
but...well, I can't see a way around it using LMDB.
A couple of other minor changes in this commit, only slightly relevant.
|
|
The rest should just be tedious copypasta and modification.
|
|
Some BlockchainDB unit testing fleshed out (and working), rudimentary
linker flag for lmdb in CMakeLists, but should probably be done
"correctly" at some point (find it on whatever system you're building on
and all that jazz).
update for rebase (warptangent 2015-01-04)
fix conflicts with upstream CMakeLists.txt files
tests/CMakeLists.txt (remove edits from original commit)
|
|
|
|
miscellaneous changes to BlockchainDB/blockchain as well, namely
replacing instances of std::list with std::vector
|
|
libglim is an Apache-licensed C++ wrapper for lmdb, and rather than
rolling our own it seems prudent to use it.
Note: lmdb is not included in it, and unless something happens as did
with libunbound, should be installed via each OS' package manager or
equivalent.
|
|
Still need to add in the new checkpointing functionality, as well as
touch up a few things, but is okay for now.
|
|
|
|
|
|
tried rebasing, tree-filter, and many other things. at this point,
the history of these files previous to this can live on
in my bc2 branch, as I'm importing them as-is to here.
|
|
f4b69d5 year updated in license (Riccardo Spagni)
|
|
|
|
|
|
|
|
|
|
583cf0a Document existing function (warptangent)
95eb944 Repeat prompt for wallet path if invalid (warptangent)
|
|
d7aafd5 wallet2::rewrite update to not require bin file (warptangent)
|
|
9b7e0a0 wallet2::load correctly initialize m_blockchain for wallet loaded without bin file (warptangent)
|
|
cfc8c55 Add simple_wallet::seed_set_language method (warptangent)
26b87df Add wallet2::verify_password method (warptangent)
|
|
b261448 revert the removal of the 'arch' flag from the release-static target (Riccardo Spagni)
a956f07 added gpg key for warptangent (Riccardo Spagni)
ac19084 revert the removal of the 'arch' flag from the release-static target (Riccardo Spagni)
|
|
|
|
Signed-off-by: Riccardo Spagni <ric@spagni.net>
|
|
|
|
Signed-off-by: Riccardo Spagni <ric@spagni.net>
|
