| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
It's not allowed to use WaitForSingleObject with _beginthread, because the thread closes its own handle before exiting.
So the wait function will either wait on an invalid handle, or on a different handle used by something else.
Or, if it starts waiting before the thread exits, the behavior is undefined according to MS: "If this handle is closed while the wait is still pending, the function's behavior is undefined."
In my test sync I observed threads getting stuck infinitely on WaitForSingleObject, and then rx_set_main_seedhash spamming new threads when RandomX seed changes again. Eventually the system ran out of resources, and monerod aborted with "Couldn't start RandomX seed thread" message.
This PR fixes it by using `_beginthreadex` instead and explicitly closing the handle when it's safe.
|
|
|
|
|
|
- Straight-forward call interface: `void rx_slow_hash(const char *seedhash, const void *data, size_t length, char *result_hash)`
- Consensus chain seed hash is now updated by calling `rx_set_main_seedhash` whenever a block is added/removed or a reorg happens
- `rx_slow_hash` will compute correct hash no matter if `rx_set_main_seedhash` was called or not (the only difference is performance)
- New environment variable `MONERO_RANDOMX_FULL_MEM` to force use the full dataset for PoW verification (faster block verification)
- When dataset is used for PoW verification, dataset updates don't stall other threads (verification is done in light mode then)
- When mining is running, PoW checks now also use dataset for faster verification
|
|
All tests were conducted on the same PC (Ryzen 5 5600X running at fixed 4.65 GHz).
Before:
test_cn_fast_hash<32> (100000 calls) - OK: 1 us/call
test_cn_fast_hash<16384> (1000 calls) - OK: 164 us/call
After:
test_cn_fast_hash<32> (100000 calls) - OK: 0 us/call
test_cn_fast_hash<16384> (1000 calls) - OK: 31 us/call
More than 5 times speedup for cn_fast_hash.
Also noticed consistent 1-2% improvement in test_construct_tx results.
|
|
Implements view tags as proposed by @UkoeHB in MRL issue
https://github.com/monero-project/research-lab/issues/73
At tx construction, the sender adds a 1-byte view tag to each
output. The view tag is derived from the sender-receiver
shared secret. When scanning for outputs, the receiver can
check the view tag for a match, in order to reduce scanning
time. When the view tag does not match, the wallet avoids the
more expensive EC operations when deriving the output public
key using the shared secret.
|
|
|
|
|
|
|
|
If allocating large pages fails, we don't try again.
This has the obvious drawback of not being able to use large pages
if they fail once.
|
|
haven't been reduced by the field order
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
include all public proof parameters in Schnorr challenges, along with hash function domain separators. Includes new randomized unit tests.
|
|
Some tools report the alignment check as UB, which seems a bit
dubious, but since the performance difference between the two
versions is minimal, I'll go with the safe version
|
|
|
|
Update copyright year to 2020
|
|
Reported by UkoeHB_ and sarang
|
|
- New flag in NOTIFY_NEW_TRANSACTION to indicate stem mode
- Stem loops detected in tx_pool.cpp
- Embargo timeout for a blackhole attack during stem phase
|
|
|
|
Don't try to allocate the dataset repeatedly if it has already failed.
|
|
|
|
* Faster cache initialization with SSSE3/AVX2
* Automatic detection of CPU capabilities in RandomX
* Fixed a possible out-of-bounds access in superscalar program generator
* Use MONERO_RANDOMX_UMASK to manually disable RandomX flags in monerod
|
|
Make sure dataset gets re-init'd if a reorg changes the epoch
|
|
We don't need to detect if the cache has changed, just always
call to set it on the VM. The call will be a no-op if the cache
hasn't changed.
|
|
Was using the wrong cache slot, and returning invalid PoW hashes to RPC clients
|
|
Support RandomX PoW algorithm
|
|
Simplify m_template initialization in miner
|
|
|
|
|
|
|
|
It lets the user add custom entropy to the PRNG.
It does this by hashing the new data and xoring the resulting
hash with the PRNG state.
|
|
|
|
The code generated is exactly the same as the direct access
one on x86_64
|
|
The value was positive rather than zero, but the caller only
checks for negative errors
|
|
NULL is valid when size is 0, but memcpy uses nonnull attributes,
so let's not poke the bear
|
|
Large amounts might run out of stack
Reported by guidov
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Enabled by setting the MONERO_USE_CNV4_JIT env var to 1
|
|
|
|
Minimalistic JIT code generator for random math sequence in CryptonightR.
Usage:
- Allocate writable and executable memory
- Call v4_generate_JIT_code with "buf" pointed to memory allocated on the previous step
- Call the generated code instead of "v4_random_math(code, r)", omit the "code" parameter
|
|
NetBSD emits:
warning: Warning: reference to the libc supplied alloca(3); this most likely will not work. Please use the compiler provided version of alloca(3), by supplying the appropriate compiler flags (e.g. not -std=c89).
and man 3 alloca says:
Normally, gcc(1) translates calls to alloca() with inlined code. This is not done when either the -ansi, -std=c89, -std=c99, or the
-std=c11 option is given and the header <alloca.h> is not included. Otherwise, (without an -ansi or -std=c* option) the glibc version of
<stdlib.h> includes <alloca.h> and that contains the lines:
#ifdef __GNUC__
#define alloca(size) __builtin_alloca (size)
#endif
It looks like alloca is a bad idea in modern C/C++, so we use
VLAs for C and std::vector for C++.
|
|
|
|
Co-Authored-By: Lee Clagett <vtnerd@users.noreply.github.com>
|
|
It introduces random integer math into the main loop.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- These functions are declared twice in slow-hash.c. Remove one of the copies.
- The declarations have the wrong return type, should be void, not int.
Function definitions here: https://github.com/monero-project/monero/blob/1e74586ee99e4bd89626d2eb4d23883cd91f0f81/src/crypto/aesb.c#L151-L180
Test plan: make release-test
|
|
|
|
17142ec9 malloc scratchpad for all supported android archs (m2049r)
|
|
|
|
|
|
|
|
|
|
Reported by QuarksLab.
|
|
Reported by QuarksLab.
|
|
This avoids problems when the caller can't deal with a zero
walue, which happens often enough that it's worth nipping the
problem in the bud.
|
|
- fix integer overflow in n_bulletproof_amounts
- check input scalars are in range
- remove use of environment variable to tweak straus performance
- do not use implementation defined signed shift for signum
|
|
|
|
Contains two modifications to improve ASIC resistance: shuffle and integer math.
Shuffle makes use of the whole 64-byte cache line instead of 16 bytes only, making Cryptonight 4 times more demanding for memory bandwidth.
Integer math adds 64:32 bit integer division followed by 64 bit integer square root, adding large and unavoidable computational latency to the main loop.
More details and performance numbers: https://github.com/SChernykh/xmr-stak-cpu/blob/master/README.md
|
|
instead of including the text, as it's not fully free to redistribute
|
|
|
|
|
|
- needed for TREZOR integration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
hash: add prehashed version cn_slow_hash_prehashed
slow-hash: let cn_slow_hash take 4th parameter for deciding prehashed or not
slow-hash: add support for prehashed version for the other 3 platforms
|
|
|
|
When #3303 was merged, a cyclic dependency chain was generated:
libdevice <- libcncrypto <- libringct <- libdevice
This was because libdevice needs access to a set of basic crypto operations
implemented in libringct such as scalarmultBase(), while libringct also needs
access to abstracted crypto operations implemented in libdevice such as
ecdhEncode(). To untangle this cyclic dependency chain, this patch splits libringct
into libringct_basic and libringct, where the basic crypto ops previously in
libringct are moved into libringct_basic. The cyclic dependency is now resolved
thanks to this separation:
libcncrypto <- libringct_basic <- libdevice <- libcryptonote_basic <- libringct
This eliminates the need for crypto_device.cpp and rctOps_device.cpp.
Also, many abstracted interfaces of hw::device such as encrypt_payment_id() and
get_subaddress_secret_key() were previously implemented in libcryptonote_basic
(cryptonote_format_utils.cpp) and were then called from hw::core::device_default,
which is odd because libdevice is supposed to be independent of libcryptonote_basic.
Therefore, those functions were moved to device_default.cpp.
|
|
|
|
|
|
|
|
|
|
This is the first variant of many, with the intent to improve
Monero's resistance to ASICs and encourage mining decentralization.
|
|
|
|
The basic approach it to delegate all sensitive data (master key, secret
ephemeral key, key derivation, ....) and related operations to the device.
As device has low memory, it does not keep itself the values
(except for view/spend keys) but once computed there are encrypted (with AES
are equivalent) and return back to monero-wallet-cli. When they need to be
manipulated by the device, they are decrypted on receive.
Moreover, using the client for storing the value in encrypted form limits
the modification in the client code. Those values are transfered from one
C-structure to another one as previously.
The code modification has been done with the wishes to be open to any
other hardware wallet. To achieve that a C++ class hw::Device has been
introduced. Two initial implementations are provided: the "default", which
remaps all calls to initial Monero code, and the "Ledger", which delegates
all calls to Ledger device.
|
|
|
|
found by h908714124
|
|
Nothing calls this with those inputs
|
|
Avoids cores being created, as they're nowadays often piped
to some call home system
|
|
|
|
|
|
|
|
ge_scalarmult_p3
ge_double_scalarmult_precomp_vartime2_p3
ge_double_scalarmult_base_vartime_p3
This makes it possible to reuse the result without having to
convert back to unsigned char[32] and back to ge types.
|
|
|
|
|
|
|
|
This does not happen when used by the monero code
|
|
Partially implements #74.
Securely erases keys from memory after they are no longer needed. Might have a
performance impact, which I haven't measured (perf measurements aren't
generally reliable on laptops).
Thanks to @stoffu for the suggestion to specialize the pod_to_hex/hex_to_pod
functions. Using overloads + SFINAE instead generalizes it so other types can
be marked as scrubbed without adding more boilerplate.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
not the actual tx secret key
|
|
Tweak temp variables and constraints. Was working before if not inlined
but newer gcc tends to inline it.
|
|
fix a cmakelist
|
|
The inline asm was lying about its parameters
|
|
|
|
|
|
CryptoNight does exactly 524,288 iterations over the scratchpad as defined in CNS008, saying 500,000 could be confusing. I know its meant to give a rough idea (around 500k) to the reader but if you are reading the code, might as well know the exact number.
|
|
|
|
|
|
|
|
|
|
- It builds but no further testing has been done.
|
|
found by coverity
|
|
View wallets do not have the spend secret key, and are thus
unable to derive key images for incoming outputs. Moreover,
a previous patch set key images to zero as a means to mark
an output as having an unknown key image, so they could be
filled in when importing key images at a later time. That
later patch caused spurious collisions. We now use public
keys to detect duplicate outputs. Public keys obtained from
the blockchain are checked to be identical to the ones
derived locally, so can't be spoofed.
|
|
|
|
|
|
|
|
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.
Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).
This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
|
|
aesb.c is already present in libcrypto as a standalone object.
Tested: builds and runs fine on armv7, static and dynamic.
|
|
More than twice as fast as plain C code. Note that both ARMv7 and
ARMv8 can be further improved with better use of NEON.
Also tweak ARMv7 multiplier
|
|
This was disabled earlier as part of diagnosing failing tests
on ARM, which turned out to be due to aliasing, fixed by
adding -fno-strict-aliasing. So, re-enabling it back.
|
|
This allows the key to be not the same for two outputs sent to
the same address (eg, if you pay yourself, and also get change
back). Also remove the key amounts lists and return parameters
since we don't actually generate random ones, so we don't need
to save them as we can recalculate them when needed if we have
the correct keys.
|
|
|
|
|
|
The implementation of mul in asm breaks 'slow-hash' test when built with
GCC 6.1.1. Disable this implementation in favor of plain C until it is
fixed.
|
|
GCC warned about this one.
|
|
This list is already defined within the function. The
removed definition was shadowed.
|
|
|
|
Btw, the warning 4200 remains disabled, but it did not get triggered
(GCC 6.1.1, ARM). But, perhaps a better way than disabling
the warning would be to do what is suggested here:
http://stackoverflow.com/questions/3350852/how-to-correctly-fix-zero-sized-array-in-struct-union-warning-c4200-without%3E
|
|
And add a thread safe version to encourage proper use
|
|
Avoids silent use of bad RNG in release builds, in case those
calls might actually fail.
Reported by smooth.
|
|
|
|
|
|
and all other associated IPC
|
|
|
|
Setting to no or 0 also works. If set, any other value enables it.
Useful for running with valgrind in cases where it fails at
properly implementing AES-NI.
|
|
|
|
About 10% faster than plain C mul128 on raspi1B
|
|
|
|
Remove trailing whitespace in same files.
|
|
|
|
|
|
(disabling it was unintentional)
|
|
crypto-ops with a version straight from Bernstein's ref 10
|
|
|
|
have bitmonero's crypto code come from bernstein et al's ref 10 code
|
|
|
|
|
|
|
|
Pros:
- smaller on the blockchain
- shorter integrated addresses
Cons:
- less sparseness
- less ability to embed actual information
The boolean argument to encrypt payment ids is now gone from the
RPC calls, since the decision is made based on the length of the
payment id passed.
|
|
Bockchain:
1. Optim: Multi-thread long-hash computation when encountering groups of blocks.
2. Optim: Cache verified txs and return result from cache instead of re-checking whenever possible.
3. Optim: Preload output-keys when encoutering groups of blocks. Sort by amount and global-index before bulk querying database and multi-thread when possible.
4. Optim: Disable double spend check on block verification, double spend is already detected when trying to add blocks.
5. Optim: Multi-thread signature computation whenever possible.
6. Patch: Disable locking (recursive mutex) on called functions from check_tx_inputs which causes slowdowns (only seems to happen on ubuntu/VMs??? Reason: TBD)
7. Optim: Removed looped full-tx hash computation when retrieving transactions from pool (???).
8. Optim: Cache difficulty/timestamps (735 blocks) for next-difficulty calculations so that only 2 db reads per new block is needed when a new block arrives (instead of 1470 reads).
Berkeley-DB:
1. Fix: 32-bit data errors causing wrong output global indices and failure to send blocks to peers (etc).
2. Fix: Unable to pop blocks on reorganize due to transaction errors.
3. Patch: Large number of transaction aborts when running multi-threaded bulk queries.
4. Patch: Insufficient locks error when running full sync.
5. Patch: Incorrect db stats when returning from an immediate exit from "pop block" operation.
6. Optim: Add bulk queries to get output global indices.
7. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
8. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
9. Optim: Added thread-safe buffers used when multi-threading bulk queries.
10. Optim: Added support for nosync/write_nosync options for improved performance (*see --db-sync-mode option for details)
11. Mod: Added checkpoint thread and auto-remove-logs option.
12. *Now usable on 32-bit systems like RPI2.
LMDB:
1. Optim: Added custom comparison for 256-bit key tables (minor speed-up, TBD: get actual effect)
2. Optim: Modified output_keys table to store public_key+unlock_time+height for single transaction lookup (vs 3)
3. Optim: Used output_keys table retrieve public_keys instead of going through output_amounts->output_txs+output_indices->txs->output:public_key
4. Optim: Added support for sync/writemap options for improved performance (*see --db-sync-mode option for details)
5. Mod: Auto resize to +1GB instead of multiplier x1.5
ETC:
1. Minor optimizations for slow-hash for ARM (RPI2). Incomplete.
2. Fix: 32-bit saturation bug when computing next difficulty on large blocks.
[PENDING ISSUES]
1. Berkely db has a very slow "pop-block" operation. This is very noticeable on the RPI2 as it sometimes takes > 10 MINUTES to pop a block during reorganization.
This does not happen very often however, most reorgs seem to take a few seconds but it possibly depends on the number of outputs present. TBD.
2. Berkeley db, possible bug "unable to allocate memory". TBD.
[NEW OPTIONS] (*Currently all enabled for testing purposes)
1. --fast-block-sync arg=[0:1] (default: 1)
a. 0 = Compute long hash per block (may take a while depending on CPU)
b. 1 = Skip long-hash and verify blocks based on embedded known good block hashes (faster, minimal CPU dependence)
2. --db-sync-mode arg=[[safe|fast|fastest]:[sync|async]:[nblocks_per_sync]] (default: fastest:async:1000)
a. safe = fdatasync/fsync (or equivalent) per stored block. Very slow, but safest option to protect against power-out/crash conditions.
b. fast/fastest = Enables asynchronous fdatasync/fsync (or equivalent). Useful for battery operated devices or STABLE systems with UPS and/or systems with battery backed write cache/solid state cache.
Fast - Write meta-data but defer data flush.
Fastest - Defer meta-data and data flush.
Sync - Flush data after nblocks_per_sync and wait.
Async - Flush data after nblocks_per_sync but do not wait for the operation to finish.
3. --prep-blocks-threads arg=[n] (default: 4 or system max threads, whichever is lower)
Max number of threads to use when computing long-hash in groups.
4. --show-time-stats arg=[0:1] (default: 1)
Show benchmark related time stats.
5. --db-auto-remove-logs arg=[0:1] (default: 1)
For berkeley-db only. Auto remove logs if enabled.
**Note: lmdb and berkeley-db have changes to the tables and are not compatible with official git head version.
At the moment, you need a full resync to use this optimized version.
[PERFORMANCE COMPARISON]
**Some figures are approximations only.
Using a baseline machine of an i7-2600K+SSD+(with full pow computation):
1. The optimized lmdb/blockhain core can process blocks up to 585K for ~1.25 hours + download time, so it usually takes 2.5 hours to sync the full chain.
2. The current head with memory can process blocks up to 585K for ~4.2 hours + download time, so it usually takes 5.5 hours to sync the full chain.
3. The current head with lmdb can process blocks up to 585K for ~32 hours + download time and usually takes 36 hours to sync the full chain.
Averate procesing times (with full pow computation):
lmdb-optimized:
1. tx_ave = 2.5 ms / tx
2. block_ave = 5.87 ms / block
memory-official-repo:
1. tx_ave = 8.85 ms / tx
2. block_ave = 19.68 ms / block
lmdb-official-repo (0f4a036437fd41a5498ee5e74e2422ea6177aa3e)
1. tx_ave = 47.8 ms / tx
2. block_ave = 64.2 ms / block
**Note: The following data denotes processing times only (does not include p2p download time)
lmdb-optimized processing times (with full pow computation):
1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 1.25 hours processing time (--db-sync-mode=fastest:async:1000).
2. Laptop, Dual-core / 4-threads U4200 (3Mb) - 4.90 hours processing time (--db-sync-mode=fastest:async:1000).
3. Embedded, Quad-core / 4-threads Z3735F (2x1Mb) - 12.0 hours processing time (--db-sync-mode=fastest:async:1000).
lmdb-optimized processing times (with per-block-checkpoint)
1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 10 minutes processing time (--db-sync-mode=fastest:async:1000).
berkeley-db optimized processing times (with full pow computation)
1. Desktop, Quad-core / 8-threads 2600k (8Mb) - 1.8 hours processing time (--db-sync-mode=fastest:async:1000).
2. RPI2. Improved from estimated 3 months(???) into 2.5 days (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
berkeley-db optimized processing times (with per-block-checkpoint)
1. RPI2. 12-15 hours (*Need 2AMP supply + Clock:1Ghz + [usb+ssd] to achieve this speed) (--db-sync-mode=fastest:async:1000).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This cleans up the CMake code and shows patterns more easily (to be
refactored in the next commit).
|
|
|
|
|
|
|
|
|
|
|
|
|
