diff options
129 files changed, 11660 insertions, 1410 deletions
diff --git a/ANONYMITY_NETWORKS.md b/ANONYMITY_NETWORKS.md new file mode 100644 index 000000000..a5f18010e --- /dev/null +++ b/ANONYMITY_NETWORKS.md @@ -0,0 +1,176 @@ +# Anonymity Networks with Monero + +Currently only Tor and I2P have been integrated into Monero. The usage of +these networks is still considered experimental - there are a few pessimistic +cases where privacy is leaked. The design is intended to maximize privacy of +the source of a transaction by broadcasting it over an anonymity network, while +relying on IPv4 for the remainder of messages to make surrounding node attacks +(via sybil) more difficult. + + +## Behavior + +If _any_ anonymity network is enabled, transactions being broadcast that lack +a valid "context" (i.e. the transaction did not come from a p2p connection), +will only be sent to peers on anonymity networks. If an anonymity network is +enabled but no peers over an anonymity network are available, an error is +logged and the transaction is kept for future broadcasting over an anonymity +network. The transaction will not be broadcast unless an anonymity connection +is made or until `monerod` is shutdown and restarted with only public +connections enabled. + + +## P2P Commands + +Only handshakes, peer timed syncs and transaction broadcast messages are +supported over anonymity networks. If one `--add-exclusive-node` p2p address +is specified, then no syncing will take place and only transaction broadcasting +can occur. It is therefore recommended that `--add-exclusive-node` be combined +with additional exclusive IPv4 address(es). + + +## Usage + +Anonymity networks have no seed nodes (the feature is still considered +experimental), so a user must specify an address. If configured properly, +additional peers can be found through typical p2p peerlist sharing. + +### Outbound Connections + +Connecting to an anonymous address requires the command line option +`--proxy` which tells `monerod` the ip/port of a socks proxy provided by a +separate process. On most systems the configuration will look like: + +> `--proxy tor,127.0.0.1:9050,10` +> `--proxy i2p,127.0.0.1:9000` + +which tells `monerod` that ".onion" p2p addresses can be forwarded to a socks +proxy at IP 127.0.0.1 port 9050 with a max of 10 outgoing connections and +".b32.i2p" p2p addresses can be forwarded to a socks proxy at IP 127.0.0.1 port +9000 with the default max outgoing connections. Since there are no seed nodes +for anonymity connections, peers must be manually specified: + +> `--add-exclusive-node rveahdfho7wo4b2m.onion:28083` +> `--add-peer rveahdfho7wo4b2m.onion:28083` + +Either option can be listed multiple times, and can specify any mix of Tor, +I2P, and IPv4 addresses. Using `--add-exclusive-node` will prevent the usage of +seed nodes on ALL networks, which will typically be undesireable. + +### Inbound Connections + +Receiving anonymity connections is done through the option +`--anonymous-inbound`. This option tells `monerod` the inbound address, network +type, and max connections: + +> `--anonymous-inbound rveahdfho7wo4b2m.onion:28083,127.0.0.1:28083,25` +> `--anonymous-inbound cmeua5767mz2q5jsaelk2rxhf67agrwuetaso5dzbenyzwlbkg2q.b32.i2p:5000,127.0.0.1:30000` + +which tells `monerod` that a max of 25 inbound Tor connections are being +received at address "rveahdfho7wo4b2m.onion:28083" and forwarded to `monerod` +localhost port 28083, and a default max I2P connections are being received at +address "cmeua5767mz2q5jsaelk2rxhf67agrwuetaso5dzbenyzwlbkg2q.b32.i2p:5000" and +forwarded to `monerod` localhost port 30000. +These addresses will be shared with outgoing peers, over the same network type, +otherwise the peer will not be notified of the peer address by the proxy. + +### Network Types + +#### Tor & I2P + +Options `--add-exclusive-node` and `--add-peer` recognize ".onion" and +".b32.i2p" addresses, and will properly forward those addresses to the proxy +provided with `--proxy tor,...` or `--proxy i2p,...`. + +Option `--anonymous-inbound` also recognizes ".onion" and ".b32.i2p" addresses, +and will automatically be sent out to outgoing Tor/I2P connections so the peer +can distribute the address to its other peers. + +##### Configuration + +Tor must be configured for hidden services. An example configuration ("torrc") +might look like: + +> HiddenServiceDir /var/lib/tor/data/monero +> HiddenServicePort 28083 127.0.0.1:28083 + +This will store key information in `/var/lib/tor/data/monero` and will forward +"Tor port" 28083 to port 28083 of ip 127.0.0.1. The file +`/usr/lib/tor/data/monero/hostname` will contain the ".onion" address for use +with `--anonymous-inbound`. + +I2P must be configured with a standard server tunnel. Configuration differs by +I2P implementation. + +## Privacy Limitations + +There are currently some techniques that could be used to _possibly_ identify +the machine that broadcast a transaction over an anonymity network. + +### Timestamps + +The peer timed sync command sends the current time in the message. This value +can be used to link an onion address to an IPv4/IPv6 address. If a peer first +sees a transaction over Tor, it could _assume_ (possibly incorrectly) that the +transaction originated from the peer. If both the Tor connection and an +IPv4/IPv6 connection have timestamps that are approximately close in value they +could be used to link the two connections. This is less likely to happen if the +system clock is fairly accurate - many peers on the Monero network should have +similar timestamps. + +#### Mitigation + +Keep the system clock accurate so that fingerprinting is more difficult. In +the future a random offset might be applied to anonymity networks so that if +the system clock is noticeably off (and therefore more fingerprintable), +linking the public IPv4/IPv6 connections with the anonymity networks will be +more difficult. + +### Bandwidth Usage + +An ISP can passively monitor `monerod` connections from a node and observe when +a transaction is sent over a Tor/I2P connection via timing analysis + size of +data sent during that timeframe. I2P should provide better protection against +this attack - its connections are not circuit based. However, if a node is +only using I2P for broadcasting Monero transactions, the total aggregate of +I2P data would also leak information. + +#### Mitigation + +There is no current mitigation for the user right now. This attack is fairly +sophisticated, and likely requires support from the internet host of a Monero +user. + +In the near future, "whitening" the amount of data sent over anonymity network +connections will be performed. An attempt will be made to make a transaction +broadcast indistinguishable from a peer timed sync command. + +### Intermittent Monero Syncing + +If a user only runs `monerod` to send a transaction then quit, this can also +be used by an ISP to link a user to a transaction. + +#### Mitigation + +Run `monerod` as often as possible to conceal when transactions are being sent. +Future versions will also have peers that first receive a transaction over an +anonymity network delay the broadcast to public peers by a randomized amount. +This will not completetely mitigate a user who syncs up sends then quits, in +part because this rule is not enforceable, so this mitigation strategy is +simply a best effort attempt. + +### Active Bandwidth Shaping + +An attacker could attempt to bandwidth shape traffic in an attempt to determine +the source of a Tor/I2P connection. There isn't great mitigation against +this, but I2P should provide better protection against this attack since +the connections are not circuit based. + +#### Mitigation + +The best mitigiation is to use I2P instead of Tor. However, I2P +has a smaller set of users (less cover traffic) and academic reviews, so there +is a tradeoff in potential isses. Also, anyone attempting this strategy really +wants to uncover a user, it seems unlikely that this would be performed against +every Tor/I2P user. + diff --git a/CMakeLists.txt b/CMakeLists.txt index f3f3e90a4..aa03b03ac 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -44,6 +44,8 @@ message(STATUS "CMake version ${CMAKE_VERSION}") project(monero) +enable_language(C ASM) + function (die msg) if (NOT WIN32) string(ASCII 27 Esc) @@ -29,6 +29,12 @@ Our researchers are available on IRC in [#monero-research-lab on Freenode](https - You can subscribe to an [announcement listserv](https://lists.getmonero.org) to get critical announcements from the Monero core team. The announcement list can be very helpful for knowing when software updates are needed. +## Translations +The CLI wallet is available in different languages. If you want to help translate it, join Pootle, our self-hosted localization platform: [translate.getmonero.org](https://translate.getmonero.org). Every translation *must* be uploaded on Pootle, pull requests directly editing the code on this repository will be closed. + + +If you need help/support/info, contact the localization workgroup. You can do so in various ways: by email (translate[at]getmonero[dot]org), on the official chat (#monero-translations). A complete list of contacts can be found on the repository of the workgroup: [monero-ecosystem/monero-translations](https://github.com/monero-ecosystem/monero-translations#contacts). + ## Build ### IMPORTANT @@ -169,6 +175,9 @@ build the library binary manually. This can be done with the following command ` Debian / Ubuntu one liner for all dependencies ``` sudo apt update && sudo apt install build-essential cmake pkg-config libboost-all-dev libssl-dev libzmq3-dev libunbound-dev libsodium-dev libunwind8-dev liblzma-dev libreadline6-dev libldns-dev libexpat1-dev doxygen graphviz libpgm-dev``` +FreeBSD one liner for required to build dependencies +```pkg install git gmake cmake pkgconf boost-libs cppzmq libsodium``` + ### Cloning the repository Clone recursively to pull-in needed submodule(s): @@ -190,7 +199,7 @@ invokes cmake commands as needed. * Change to the root of the source code directory, change to the most recent release branch, and build: cd monero - git checkout v0.13.0.4 + git checkout release-v0.13 make *Optional*: If your machine has several cores and enough memory, enable @@ -254,7 +263,7 @@ Tested on a Raspberry Pi Zero with a clean install of minimal Raspbian Stretch ( ``` git clone https://github.com/monero-project/monero.git cd monero - git checkout tags/v0.13.0.4 + git checkout tags/release-v0.13 ``` * Build: ``` @@ -351,9 +360,9 @@ application. cd monero -* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'v0.13.0.0'. If you dont care about the version and just want binaries from master, skip this step: +* If you would like a specific [version/tag](https://github.com/monero-project/monero/tags), do a git checkout for that version. eg. 'release-0.13'. If you dont care about the version and just want binaries from master, skip this step: - git checkout v0.13.0.4 + git checkout release-v0.13 * If you are on a 64-bit system, run: @@ -377,7 +386,7 @@ application. ### On FreeBSD: -The project can be built from scratch by following instructions for Linux above. If you are running monero in a jail you need to add the flag: `allow.sysvipc=1` to your jail configuration, otherwise lmdb will throw the error message: `Failed to open lmdb environment: Function not implemented`. +The project can be built from scratch by following instructions for Linux above(but use `gmake` instead of `make`). If you are running monero in a jail you need to add the flag: `allow.sysvipc=1` to your jail configuration, otherwise lmdb will throw the error message: `Failed to open lmdb environment: Function not implemented`. We expect to add Monero into the ports tree in the near future, which will aid in managing installations using ports or packages. @@ -615,6 +624,12 @@ See [README.i18n.md](README.i18n.md). ## Using Tor +> There is a new, still experimental, [integration with Tor](ANONYMITY_NETWORKS.md). The +> feature allows connecting over IPv4 and Tor simulatenously - IPv4 is used for +> relaying blocks and relaying transactions received by peers whereas Tor is +> used solely for relaying transactions received over local RPC. This provides +> privacy and better protection against surrounding node (sybil) attacks. + While Monero isn't made to integrate with Tor, it can be used wrapped with torsocks, by setting the following configuration parameters and environment variables: diff --git a/contrib/epee/include/math_helper.h b/contrib/epee/include/math_helper.h index e22e8ee6e..35b649972 100644 --- a/contrib/epee/include/math_helper.h +++ b/contrib/epee/include/math_helper.h @@ -243,6 +243,7 @@ namespace math_helper present = present << 32; present |= fileTime.dwLowDateTime; present /= 10; // mic-sec + return present; #else struct timeval tv; gettimeofday(&tv, NULL); diff --git a/contrib/epee/include/net/abstract_tcp_server2.h b/contrib/epee/include/net/abstract_tcp_server2.h index e6b2755af..643b93b87 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.h +++ b/contrib/epee/include/net/abstract_tcp_server2.h @@ -41,10 +41,12 @@ #include <boost/noncopyable.hpp> #include <boost/shared_ptr.hpp> #include <atomic> +#include <cassert> #include <map> #include <memory> #include <boost/asio.hpp> +#include <boost/asio/ssl.hpp> #include <boost/array.hpp> #include <boost/noncopyable.hpp> #include <boost/shared_ptr.hpp> @@ -87,22 +89,40 @@ namespace net_utils { public: typedef typename t_protocol_handler::connection_context t_connection_context; + + struct shared_state : socket_stats + { + shared_state() + : socket_stats(), pfilter(nullptr), config() + {} + + i_connection_filter* pfilter; + typename t_protocol_handler::config_type config; + }; + /// Construct a connection with the given io_service. - explicit connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter * &pfilter - ,t_connection_type connection_type); + boost::shared_ptr<shared_state> state, + t_connection_type connection_type, + epee::net_utils::ssl_support_t ssl_support, + ssl_context_t &ssl_context); + + explicit connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type, + epee::net_utils::ssl_support_t ssl_support, + ssl_context_t &ssl_context); + + virtual ~connection() noexcept(false); - /// Get the socket associated with the connection. - boost::asio::ip::tcp::socket& socket(); /// Start the first asynchronous operation for the connection. bool start(bool is_income, bool is_multithreaded); + // `real_remote` is the actual endpoint (if connection is to proxy, etc.) + bool start(bool is_income, bool is_multithreaded, network_address real_remote); + void get_context(t_connection_context& context_){context_ = context;} void call_back_starter(); @@ -128,6 +148,10 @@ namespace net_utils //------------------------------------------------------ boost::shared_ptr<connection<t_protocol_handler> > safe_shared_from_this(); bool shutdown(); + /// Handle completion of a receive operation. + void handle_receive(const boost::system::error_code& e, + std::size_t bytes_transferred); + /// Handle completion of a read operation. void handle_read(const boost::system::error_code& e, std::size_t bytes_transferred); @@ -145,10 +169,9 @@ namespace net_utils /// Buffer for incoming data. boost::array<char, 8192> buffer_; - //boost::array<char, 1024> buffer_; + size_t buffer_ssl_init_fill; t_connection_context context; - i_connection_filter* &m_pfilter; // TODO what do they mean about wait on destructor?? --rfree : //this should be the last one, because it could be wait on destructor, while other activities possible on other threads @@ -185,6 +208,13 @@ namespace net_utils class boosted_tcp_server : private boost::noncopyable { + enum try_connect_result_t + { + CONNECT_SUCCESS, + CONNECT_FAILURE, + CONNECT_NO_SSL, + }; + public: typedef boost::shared_ptr<connection<t_protocol_handler> > connection_ptr; typedef typename t_protocol_handler::connection_context t_connection_context; @@ -198,8 +228,8 @@ namespace net_utils std::map<std::string, t_connection_type> server_type_map; void create_server_type_map(); - bool init_server(uint32_t port, const std::string address = "0.0.0.0"); - bool init_server(const std::string port, const std::string& address = "0.0.0.0"); + bool init_server(uint32_t port, const std::string address = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, const std::pair<std::string, std::string> &private_key_and_certificate_path = std::make_pair(std::string(), std::string()), const std::list<std::string> &allowed_certificates = {}, bool allow_any_cert = false); + bool init_server(const std::string port, const std::string& address = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, const std::pair<std::string, std::string> &private_key_and_certificate_path = std::make_pair(std::string(), std::string()), const std::list<std::string> &allowed_certificates = {}, bool allow_any_cert = false); /// Run the server's io_service loop. bool run_server(size_t threads_count, bool wait = true, const boost::thread::attributes& attrs = boost::thread::attributes()); @@ -210,7 +240,9 @@ namespace net_utils /// Stop the server. void send_stop_signal(); - bool is_stop_signal_sent(); + bool is_stop_signal_sent() const noexcept { return m_stop_signal_sent; }; + + const std::atomic<bool>& get_stop_signal() const noexcept { return m_stop_signal_sent; } void set_threads_prefix(const std::string& prefix_name); @@ -220,17 +252,29 @@ namespace net_utils void set_connection_filter(i_connection_filter* pfilter); - bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0"); + void set_default_remote(epee::net_utils::network_address remote) + { + default_remote = std::move(remote); + } + + bool add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote, epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect); + try_connect_result_t try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout, epee::net_utils::ssl_support_t ssl_support); + bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect); template<class t_callback> - bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0"); + bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0", epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect); - typename t_protocol_handler::config_type& get_config_object(){return m_config;} + typename t_protocol_handler::config_type& get_config_object() + { + assert(m_state != nullptr); // always set in constructor + return m_state->config; + } int get_binded_port(){return m_port;} long get_connections_count() const { - auto connections_count = (m_sock_count > 0) ? (m_sock_count - 1) : 0; // Socket count minus listening socket + assert(m_state != nullptr); // always set in constructor + auto connections_count = m_state->sock_count > 0 ? (m_state->sock_count - 1) : 0; // Socket count minus listening socket return connections_count; } @@ -292,9 +336,6 @@ namespace net_utils return true; } - protected: - typename t_protocol_handler::config_type m_config; - private: /// Run the server's io_service loop. bool worker_thread(); @@ -303,21 +344,21 @@ namespace net_utils bool is_thread_worker(); + const boost::shared_ptr<typename connection<t_protocol_handler>::shared_state> m_state; + /// The io_service used to perform asynchronous operations. std::unique_ptr<boost::asio::io_service> m_io_service_local_instance; boost::asio::io_service& io_service_; /// Acceptor used to listen for incoming connections. boost::asio::ip::tcp::acceptor acceptor_; + epee::net_utils::network_address default_remote; std::atomic<bool> m_stop_signal_sent; uint32_t m_port; - std::atomic<long> m_sock_count; - std::atomic<long> m_sock_number; std::string m_address; std::string m_thread_name_prefix; //TODO: change to enum server_type, now used size_t m_threads_count; - i_connection_filter* m_pfilter; std::vector<boost::shared_ptr<boost::thread> > m_threads; boost::thread::id m_main_thread_id; critical_section m_threads_lock; @@ -331,6 +372,9 @@ namespace net_utils boost::mutex connections_mutex; std::set<connection_ptr> connections_; + ssl_context_t m_ssl_context; + std::list<std::string> m_allowed_certificates; + }; // class <>boosted_tcp_server diff --git a/contrib/epee/include/net/abstract_tcp_server2.inl b/contrib/epee/include/net/abstract_tcp_server2.inl index 457ee2dd4..7a3abe9e9 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.inl +++ b/contrib/epee/include/net/abstract_tcp_server2.inl @@ -40,6 +40,7 @@ #include <boost/asio/deadline_timer.hpp> #include <boost/date_time/posix_time/posix_time.hpp> // TODO #include <boost/thread/condition_variable.hpp> // TODO +#include <boost/make_shared.hpp> #include "warnings.h" #include "string_tools.h" #include "misc_language.h" @@ -62,6 +63,13 @@ namespace epee { namespace net_utils { + template<typename T> + T& check_and_get(boost::shared_ptr<T>& ptr) + { + CHECK_AND_ASSERT_THROW_MES(bool(ptr), "shared_state cannot be null"); + return *ptr; + } + /************************************************************************/ /* */ /************************************************************************/ @@ -69,25 +77,35 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> connection<t_protocol_handler>::connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter* &pfilter - ,t_connection_type connection_type + boost::shared_ptr<shared_state> state, + t_connection_type connection_type, + epee::net_utils::ssl_support_t ssl_support, + ssl_context_t &ssl_context + ) + : connection(boost::asio::ip::tcp::socket{io_service}, std::move(state), connection_type, ssl_support, ssl_context) + { + } + + template<class t_protocol_handler> + connection<t_protocol_handler>::connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type, + epee::net_utils::ssl_support_t ssl_support, + ssl_context_t &ssl_context ) : - connection_basic(io_service, ref_sock_count, sock_number), - m_protocol_handler(this, config, context), - m_pfilter( pfilter ), + connection_basic(std::move(sock), state, ssl_support, ssl_context), + m_protocol_handler(this, check_and_get(state).config, context), m_connection_type( connection_type ), m_throttle_speed_in("speed_in", "throttle_speed_in"), m_throttle_speed_out("speed_out", "throttle_speed_out"), - m_timer(io_service), + m_timer(socket_.get_io_service()), m_local(false), m_ready_to_close(false) { MDEBUG("test, connection constructor set m_connection_type="<<m_connection_type); } + PRAGMA_WARNING_DISABLE_VS(4355) //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -95,17 +113,11 @@ PRAGMA_WARNING_DISABLE_VS(4355) { if(!m_was_shutdown) { - _dbg3("[sock " << socket_.native_handle() << "] Socket destroyed without shutdown."); + _dbg3("[sock " << socket().native_handle() << "] Socket destroyed without shutdown."); shutdown(); } - _dbg3("[sock " << socket_.native_handle() << "] Socket destroyed"); - } - //--------------------------------------------------------------------------------- - template<class t_protocol_handler> - boost::asio::ip::tcp::socket& connection<t_protocol_handler>::socket() - { - return socket_; + _dbg3("[sock " << socket().native_handle() << "] Socket destroyed"); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -127,36 +139,47 @@ PRAGMA_WARNING_DISABLE_VS(4355) { TRY_ENTRY(); + boost::system::error_code ec; + auto remote_ep = socket().remote_endpoint(ec); + CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); + CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); + + const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; + return start(is_income, is_multithreaded, ipv4_network_address{uint32_t(ip_), remote_ep.port()}); + CATCH_ENTRY_L0("connection<t_protocol_handler>::start()", false); + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> + bool connection<t_protocol_handler>::start(bool is_income, bool is_multithreaded, network_address real_remote) + { + TRY_ENTRY(); + // Use safe_shared_from_this, because of this is public method and it can be called on the object being deleted auto self = safe_shared_from_this(); if(!self) return false; m_is_multithreaded = is_multithreaded; - - boost::system::error_code ec; - auto remote_ep = socket_.remote_endpoint(ec); - CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); - CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); - - auto local_ep = socket_.local_endpoint(ec); - CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get local endpoint: " << ec.message() << ':' << ec.value()); - - context = boost::value_initialized<t_connection_context>(); - const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; - m_local = epee::net_utils::is_ip_loopback(ip_) || epee::net_utils::is_ip_local(ip_); + m_local = real_remote.is_loopback() || real_remote.is_local(); // create a random uuid, we don't need crypto strength here const boost::uuids::uuid random_uuid = boost::uuids::random_generator()(); - context.set_details(random_uuid, epee::net_utils::ipv4_network_address(ip_, remote_ep.port()), is_income); + context = t_connection_context{}; + bool ssl = m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled; + context.set_details(random_uuid, std::move(real_remote), is_income, ssl); + + boost::system::error_code ec; + auto local_ep = socket().local_endpoint(ec); + CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get local endpoint: " << ec.message() << ':' << ec.value()); + _dbg3("[sock " << socket_.native_handle() << "] new connection from " << print_connection_context_short(context) << " to " << local_ep.address().to_string() << ':' << local_ep.port() << - ", total sockets objects " << m_ref_sock_count); + ", total sockets objects " << get_stats().sock_count); - if(m_pfilter && !m_pfilter->is_remote_host_allowed(context.m_remote_address)) + if(static_cast<shared_state&>(get_stats()).pfilter && !static_cast<shared_state&>(get_stats()).pfilter->is_remote_host_allowed(context.m_remote_address)) { - _dbg2("[sock " << socket_.native_handle() << "] host denied " << context.m_remote_address.host_str() << ", shutdowning connection"); + _dbg2("[sock " << socket().native_handle() << "] host denied " << context.m_remote_address.host_str() << ", shutdowning connection"); close(); return false; } @@ -168,11 +191,21 @@ PRAGMA_WARNING_DISABLE_VS(4355) reset_timer(get_default_timeout(), false); - socket_.async_read_some(boost::asio::buffer(buffer_), - strand_.wrap( - boost::bind(&connection<t_protocol_handler>::handle_read, self, - boost::asio::placeholders::error, - boost::asio::placeholders::bytes_transferred))); + // first read on the raw socket to detect SSL for the server + buffer_ssl_init_fill = 0; + if (is_income && m_ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_disabled) + socket().async_receive(boost::asio::buffer(buffer_), + boost::asio::socket_base::message_peek, + strand_.wrap( + boost::bind(&connection<t_protocol_handler>::handle_receive, self, + boost::asio::placeholders::error, + boost::asio::placeholders::bytes_transferred))); + else + async_read_some(boost::asio::buffer(buffer_), + strand_.wrap( + boost::bind(&connection<t_protocol_handler>::handle_read, self, + boost::asio::placeholders::error, + boost::asio::placeholders::bytes_transferred))); #if !defined(_WIN32) || !defined(__i686) // not supported before Windows7, too lazy for runtime check // Just exclude for 32bit windows builds @@ -180,12 +213,12 @@ PRAGMA_WARNING_DISABLE_VS(4355) int tos = get_tos_flag(); boost::asio::detail::socket_option::integer< IPPROTO_IP, IP_TOS > optionTos( tos ); - socket_.set_option( optionTos ); + socket().set_option( optionTos ); //_dbg1("Set ToS flag to " << tos); #endif boost::asio::ip::tcp::no_delay noDelayOption(false); - socket_.set_option(noDelayOption); + socket().set_option(noDelayOption); return true; @@ -210,7 +243,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boost::asio::io_service& connection<t_protocol_handler>::get_io_service() { - return socket_.get_io_service(); + return socket().get_io_service(); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -222,9 +255,9 @@ PRAGMA_WARNING_DISABLE_VS(4355) auto self = safe_shared_from_this(); if(!self) return false; - //_dbg3("[sock " << socket_.native_handle() << "] add_ref, m_peer_number=" << mI->m_peer_number); + //_dbg3("[sock " << socket().native_handle() << "] add_ref, m_peer_number=" << mI->m_peer_number); CRITICAL_REGION_LOCAL(self->m_self_refs_lock); - //_dbg3("[sock " << socket_.native_handle() << "] add_ref 2, m_peer_number=" << mI->m_peer_number); + //_dbg3("[sock " << socket().native_handle() << "] add_ref 2, m_peer_number=" << mI->m_peer_number); if(m_was_shutdown) return false; ++m_reference_count; @@ -238,9 +271,9 @@ PRAGMA_WARNING_DISABLE_VS(4355) { TRY_ENTRY(); boost::shared_ptr<connection<t_protocol_handler> > back_connection_copy; - LOG_TRACE_CC(context, "[sock " << socket_.native_handle() << "] release"); + LOG_TRACE_CC(context, "[sock " << socket().native_handle() << "] release"); CRITICAL_REGION_BEGIN(m_self_refs_lock); - CHECK_AND_ASSERT_MES(m_reference_count, false, "[sock " << socket_.native_handle() << "] m_reference_count already at 0 at connection<t_protocol_handler>::release() call"); + CHECK_AND_ASSERT_MES(m_reference_count, false, "[sock " << socket().native_handle() << "] m_reference_count already at 0 at connection<t_protocol_handler>::release() call"); // is this the last reference? if (--m_reference_count == 0) { // move the held reference to a local variable, keeping the object alive until the function terminates @@ -266,7 +299,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) std::string address, port; boost::system::error_code e; - boost::asio::ip::tcp::endpoint endpoint = socket_.remote_endpoint(e); + boost::asio::ip::tcp::endpoint endpoint = socket().remote_endpoint(e); if (e) { address = "<not connected>"; @@ -278,8 +311,8 @@ PRAGMA_WARNING_DISABLE_VS(4355) port = boost::lexical_cast<std::string>(endpoint.port()); } MDEBUG(" connection type " << to_string( m_connection_type ) << " " - << socket_.local_endpoint().address().to_string() << ":" << socket_.local_endpoint().port() - << " <--> " << address << ":" << port); + << socket().local_endpoint().address().to_string() << ":" << socket().local_endpoint().port() + << " <--> " << context.m_remote_address.str() << " (via " << address << ":" << port << ")"); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -287,7 +320,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) std::size_t bytes_transferred) { TRY_ENTRY(); - //_info("[sock " << socket_.native_handle() << "] Async read calledback."); + //_info("[sock " << socket().native_handle() << "] Async read calledback."); if (!e) { @@ -323,7 +356,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) } while(delay > 0); } // any form of sleeping - //_info("[sock " << socket_.native_handle() << "] RECV " << bytes_transferred); + //_info("[sock " << socket().native_handle() << "] RECV " << bytes_transferred); logger_handle_net_read(bytes_transferred); context.m_last_recv = time(NULL); context.m_recv_cnt += bytes_transferred; @@ -331,7 +364,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) bool recv_res = m_protocol_handler.handle_recv(buffer_.data(), bytes_transferred); if(!recv_res) { - //_info("[sock " << socket_.native_handle() << "] protocol_want_close"); + //_info("[sock " << socket().native_handle() << "] protocol_want_close"); //some error in protocol, protocol handler ask to close connection boost::interprocess::ipcdetail::atomic_write32(&m_want_close_connection, 1); @@ -345,24 +378,24 @@ PRAGMA_WARNING_DISABLE_VS(4355) }else { reset_timer(get_timeout_from_bytes_read(bytes_transferred), false); - socket_.async_read_some(boost::asio::buffer(buffer_), + async_read_some(boost::asio::buffer(buffer_), strand_.wrap( boost::bind(&connection<t_protocol_handler>::handle_read, connection<t_protocol_handler>::shared_from_this(), boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred))); - //_info("[sock " << socket_.native_handle() << "]Async read requested."); + //_info("[sock " << socket().native_handle() << "]Async read requested."); } }else { - _dbg3("[sock " << socket_.native_handle() << "] Some not success at read: " << e.message() << ':' << e.value()); + _dbg3("[sock " << socket().native_handle() << "] Some not success at read: " << e.message() << ':' << e.value()); if(e.value() != 2) { - _dbg3("[sock " << socket_.native_handle() << "] Some problems at read: " << e.message() << ':' << e.value()); + _dbg3("[sock " << socket().native_handle() << "] Some problems at read: " << e.message() << ':' << e.value()); shutdown(); } else { - _dbg3("[sock " << socket_.native_handle() << "] peer closed connection"); + _dbg3("[sock " << socket().native_handle() << "] peer closed connection"); if (m_ready_to_close) shutdown(); } @@ -376,13 +409,85 @@ PRAGMA_WARNING_DISABLE_VS(4355) } //--------------------------------------------------------------------------------- template<class t_protocol_handler> + void connection<t_protocol_handler>::handle_receive(const boost::system::error_code& e, + std::size_t bytes_transferred) + { + TRY_ENTRY(); + if (e) + { + // offload the error case + handle_read(e, bytes_transferred); + return; + } + + reset_timer(get_timeout_from_bytes_read(bytes_transferred), false); + + buffer_ssl_init_fill += bytes_transferred; + if (buffer_ssl_init_fill <= get_ssl_magic_size()) + { + socket().async_receive(boost::asio::buffer(buffer_.data() + buffer_ssl_init_fill, buffer_.size() - buffer_ssl_init_fill), + boost::asio::socket_base::message_peek, + strand_.wrap( + boost::bind(&connection<t_protocol_handler>::handle_receive, connection<t_protocol_handler>::shared_from_this(), + boost::asio::placeholders::error, + boost::asio::placeholders::bytes_transferred))); + return; + } + + // detect SSL + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) + { + if (is_ssl((const unsigned char*)buffer_.data(), buffer_ssl_init_fill)) + { + MDEBUG("That looks like SSL"); + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_enabled; // read/write to the SSL socket + } + else + { + MDEBUG("That does not look like SSL"); + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; // read/write to the raw socket + } + } + + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + { + // Handshake + if (!handshake(boost::asio::ssl::stream_base::server)) + { + MERROR("SSL handshake failed"); + boost::interprocess::ipcdetail::atomic_write32(&m_want_close_connection, 1); + bool do_shutdown = false; + CRITICAL_REGION_BEGIN(m_send_que_lock); + if(!m_send_que.size()) + do_shutdown = true; + CRITICAL_REGION_END(); + if(do_shutdown) + shutdown(); + return; + } + } + + async_read_some(boost::asio::buffer(buffer_), + strand_.wrap( + boost::bind(&connection<t_protocol_handler>::handle_read, connection<t_protocol_handler>::shared_from_this(), + boost::asio::placeholders::error, + boost::asio::placeholders::bytes_transferred))); + + // If an error occurs then no new asynchronous operations are started. This + // means that all shared_ptr references to the connection object will + // disappear and the object will be destroyed automatically after this + // handler returns. The connection class's destructor closes the socket. + CATCH_ENTRY_L0("connection<t_protocol_handler>::handle_receive", void()); + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> bool connection<t_protocol_handler>::call_run_once_service_io() { TRY_ENTRY(); if(!m_is_multithreaded) { //single thread model, we can wait in blocked call - size_t cnt = socket_.get_io_service().run_one(); + size_t cnt = socket().get_io_service().run_one(); if(!cnt)//service is going to quit return false; }else @@ -392,7 +497,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) //if no handlers were called //TODO: Maybe we need to have have critical section + event + callback to upper protocol to //ask it inside(!) critical region if we still able to go in event wait... - size_t cnt = socket_.get_io_service().poll_one(); + size_t cnt = socket().get_io_service().poll_one(); if(!cnt) misc_utils::sleep_no_w(1); } @@ -501,7 +606,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) context.m_max_speed_up = std::max(context.m_max_speed_up, context.m_current_speed_up); } - //_info("[sock " << socket_.native_handle() << "] SEND " << cb); + //_info("[sock " << socket().native_handle() << "] SEND " << cb); context.m_last_send = time(NULL); context.m_send_cnt += cb; //some data should be wrote to stream @@ -546,7 +651,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) MDEBUG("do_send_chunk() NOW just queues: packet="<<size_now<<" B, is added to queue-size="<<m_send_que.size()); //do_send_handler_delayed( ptr , size_now ); // (((H))) // empty function - LOG_TRACE_CC(context, "[sock " << socket_.native_handle() << "] Async send requested " << m_send_que.front().size()); + LOG_TRACE_CC(context, "[sock " << socket().native_handle() << "] Async send requested " << m_send_que.front().size()); } else { // no active operation @@ -564,14 +669,14 @@ PRAGMA_WARNING_DISABLE_VS(4355) CHECK_AND_ASSERT_MES( size_now == m_send_que.front().size(), false, "Unexpected queue size"); reset_timer(get_default_timeout(), false); - boost::asio::async_write(socket_, boost::asio::buffer(m_send_que.front().data(), size_now ) , + async_write(boost::asio::buffer(m_send_que.front().data(), size_now ) , //strand_.wrap( boost::bind(&connection<t_protocol_handler>::handle_write, self, _1, _2) //) ); //_dbg3("(chunk): " << size_now); //logger_handle_net_write(size_now); - //_info("[sock " << socket_.native_handle() << "] Async send requested " << m_send_que.front().size()); + //_info("[sock " << socket().native_handle() << "] Async send requested " << m_send_que.front().size()); } //do_send_handler_stop( ptr , cb ); // empty function @@ -656,7 +761,8 @@ PRAGMA_WARNING_DISABLE_VS(4355) // Initiate graceful connection closure. m_timer.cancel(); boost::system::error_code ignored_ec; - socket_.shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec); + socket().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec); + socket().close(); if (!m_host.empty()) { try { host_count(m_host, -1); } catch (...) { /* ignore */ } @@ -674,7 +780,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) auto self = safe_shared_from_this(); if(!self) return false; - //_info("[sock " << socket_.native_handle() << "] Que Shutdown called."); + //_info("[sock " << socket().native_handle() << "] Que Shutdown called."); m_timer.cancel(); size_t send_que_size = 0; CRITICAL_REGION_BEGIN(m_send_que_lock); @@ -709,11 +815,11 @@ PRAGMA_WARNING_DISABLE_VS(4355) void connection<t_protocol_handler>::handle_write(const boost::system::error_code& e, size_t cb) { TRY_ENTRY(); - LOG_TRACE_CC(context, "[sock " << socket_.native_handle() << "] Async send calledback " << cb); + LOG_TRACE_CC(context, "[sock " << socket().native_handle() << "] Async send calledback " << cb); if (e) { - _dbg1("[sock " << socket_.native_handle() << "] Some problems at write: " << e.message() << ':' << e.value()); + _dbg1("[sock " << socket().native_handle() << "] Some problems at write: " << e.message() << ':' << e.value()); shutdown(); return; } @@ -728,7 +834,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) CRITICAL_REGION_BEGIN(m_send_que_lock); if(m_send_que.empty()) { - _erro("[sock " << socket_.native_handle() << "] m_send_que.size() == 0 at handle_write!"); + _erro("[sock " << socket().native_handle() << "] m_send_que.size() == 0 at handle_write!"); return; } @@ -748,11 +854,11 @@ PRAGMA_WARNING_DISABLE_VS(4355) if (speed_limit_is_enabled()) do_send_handler_write_from_queue(e, m_send_que.front().size() , m_send_que.size()); // (((H))) CHECK_AND_ASSERT_MES( size_now == m_send_que.front().size(), void(), "Unexpected queue size"); - boost::asio::async_write(socket_, boost::asio::buffer(m_send_que.front().data(), size_now) , - // strand_.wrap( - boost::bind(&connection<t_protocol_handler>::handle_write, connection<t_protocol_handler>::shared_from_this(), _1, _2) - // ) - ); + async_write(boost::asio::buffer(m_send_que.front().data(), size_now) , + // strand_.wrap( + boost::bind(&connection<t_protocol_handler>::handle_write, connection<t_protocol_handler>::shared_from_this(), _1, _2) + // ) + ); //_dbg3("(normal)" << size_now); } CRITICAL_REGION_END(); @@ -784,14 +890,17 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server( t_connection_type connection_type ) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), m_io_service_local_instance(new boost::asio::io_service()), io_service_(*m_io_service_local_instance.get()), acceptor_(io_service_), + default_remote(), m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + m_threads_count(0), + m_thread_index(0), m_connection_type( connection_type ), - new_connection_() + new_connection_(), + m_ssl_context({boost::asio::ssl::context(boost::asio::ssl::context::sslv23), {}}) { create_server_type_map(); m_thread_name_prefix = "NET"; @@ -799,13 +908,16 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server(boost::asio::io_service& extarnal_io_service, t_connection_type connection_type) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), io_service_(extarnal_io_service), acceptor_(io_service_), - m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + default_remote(), + m_stop_signal_sent(false), m_port(0), + m_threads_count(0), + m_thread_index(0), m_connection_type(connection_type), - new_connection_() + new_connection_(), + m_ssl_context({boost::asio::ssl::context(boost::asio::ssl::context::sslv23), {}}) { create_server_type_map(); m_thread_name_prefix = "NET"; @@ -827,12 +939,14 @@ PRAGMA_WARNING_DISABLE_VS(4355) } //--------------------------------------------------------------------------------- template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::init_server(uint32_t port, const std::string address) + bool boosted_tcp_server<t_protocol_handler>::init_server(uint32_t port, const std::string address, epee::net_utils::ssl_support_t ssl_support, const std::pair<std::string, std::string> &private_key_and_certificate_path, const std::list<std::string> &allowed_certificates, bool allow_any_cert) { TRY_ENTRY(); m_stop_signal_sent = false; m_port = port; m_address = address; + if (ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_disabled) + m_ssl_context = create_ssl_context(private_key_and_certificate_path, allowed_certificates, allow_any_cert); // Open the acceptor with the option to reuse the address (i.e. SO_REUSEADDR). boost::asio::ip::tcp::resolver resolver(io_service_); boost::asio::ip::tcp::resolver::query query(address, boost::lexical_cast<std::string>(port), boost::asio::ip::tcp::resolver::query::canonical_name); @@ -844,7 +958,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) boost::asio::ip::tcp::endpoint binded_endpoint = acceptor_.local_endpoint(); m_port = binded_endpoint.port(); MDEBUG("start accept"); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, ssl_support, m_ssl_context)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -866,7 +980,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) PUSH_WARNINGS DISABLE_GCC_WARNING(maybe-uninitialized) template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::init_server(const std::string port, const std::string& address) + bool boosted_tcp_server<t_protocol_handler>::init_server(const std::string port, const std::string& address, epee::net_utils::ssl_support_t ssl_support, const std::pair<std::string, std::string> &private_key_and_certificate_path, const std::list<std::string> &allowed_certificates, bool allow_any_cert) { uint32_t p = 0; @@ -874,7 +988,7 @@ DISABLE_GCC_WARNING(maybe-uninitialized) MERROR("Failed to convert port no = " << port); return false; } - return this->init_server(p, address); + return this->init_server(p, address, ssl_support, private_key_and_certificate_path, allowed_certificates, allow_any_cert); } POP_WARNINGS //--------------------------------------------------------------------------------- @@ -922,7 +1036,8 @@ POP_WARNINGS template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::set_connection_filter(i_connection_filter* pfilter) { - m_pfilter = pfilter; + assert(m_state != nullptr); // always set in constructor + m_state->pfilter = pfilter; } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -1030,12 +1145,6 @@ POP_WARNINGS } //--------------------------------------------------------------------------------- template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::is_stop_signal_sent() - { - return m_stop_signal_sent; - } - //--------------------------------------------------------------------------------- - template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::handle_accept(const boost::system::error_code& e) { MDEBUG("handle_accept"); @@ -1044,11 +1153,18 @@ POP_WARNINGS if (!e) { if (m_connection_type == e_connection_type_RPC) { - MDEBUG("New server for RPC connections"); + const char *ssl_message = "unknown"; + switch (new_connection_->get_ssl_support()) + { + case epee::net_utils::ssl_support_t::e_ssl_support_disabled: ssl_message = "disabled"; break; + case epee::net_utils::ssl_support_t::e_ssl_support_enabled: ssl_message = "enabled"; break; + case epee::net_utils::ssl_support_t::e_ssl_support_autodetect: ssl_message = "autodetection"; break; + } + MDEBUG("New server for RPC connections, SSL " << ssl_message); new_connection_->setRpcStation(); // hopefully this is not needed actually } connection_ptr conn(std::move(new_connection_)); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, conn->get_ssl_support(), m_ssl_context)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -1056,7 +1172,16 @@ POP_WARNINGS boost::asio::socket_base::keep_alive opt(true); conn->socket().set_option(opt); - conn->start(true, 1 < m_threads_count); + bool res; + if (default_remote.get_type_id() == net_utils::address_type::invalid) + res = conn->start(true, 1 < m_threads_count); + else + res = conn->start(true, 1 < m_threads_count, default_remote); + if (!res) + { + conn->cancel(); + return; + } conn->save_dbg_log(); return; } @@ -1071,43 +1196,40 @@ POP_WARNINGS } // error path, if e or exception - _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_state->sock_count); misc_utils::sleep_no_w(100); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, new_connection_->get_ssl_support(), m_ssl_context)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip) + bool boosted_tcp_server<t_protocol_handler>::add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote, epee::net_utils::ssl_support_t ssl_support) { - TRY_ENTRY(); - - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); - connections_mutex.lock(); - connections_.insert(new_connection_l); - MDEBUG("connections_ size now " << connections_.size()); - connections_mutex.unlock(); - epee::misc_utils::auto_scope_leave_caller scope_exit_handler = epee::misc_utils::create_scope_leave_handler([&](){ CRITICAL_REGION_LOCAL(connections_mutex); connections_.erase(new_connection_l); }); - boost::asio::ip::tcp::socket& sock_ = new_connection_l->socket(); - - ////////////////////////////////////////////////////////////////////////// - boost::asio::ip::tcp::resolver resolver(io_service_); - boost::asio::ip::tcp::resolver::query query(boost::asio::ip::tcp::v4(), adr, port, boost::asio::ip::tcp::resolver::query::canonical_name); - boost::asio::ip::tcp::resolver::iterator iterator = resolver.resolve(query); - boost::asio::ip::tcp::resolver::iterator end; - if(iterator == end) + if(std::addressof(get_io_service()) == std::addressof(sock.get_io_service())) { - _erro("Failed to resolve " << adr); - return false; + connection_ptr conn(new connection<t_protocol_handler>(std::move(sock), m_state, m_connection_type, ssl_support, m_ssl_context)); + if(conn->start(false, 1 < m_threads_count, std::move(real_remote))) + { + conn->get_context(out); + conn->save_dbg_log(); + return true; + } } - ////////////////////////////////////////////////////////////////////////// - + else + { + MWARNING(out << " was not added, socket/io_service mismatch"); + } + return false; + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> + typename boosted_tcp_server<t_protocol_handler>::try_connect_result_t boosted_tcp_server<t_protocol_handler>::try_connect(connection_ptr new_connection_l, const std::string& adr, const std::string& port, boost::asio::ip::tcp::socket &sock_, const boost::asio::ip::tcp::endpoint &remote_endpoint, const std::string &bind_ip, uint32_t conn_timeout, epee::net_utils::ssl_support_t ssl_support) + { + TRY_ENTRY(); - //boost::asio::ip::tcp::endpoint remote_endpoint(boost::asio::ip::address::from_string(addr.c_str()), port); - boost::asio::ip::tcp::endpoint remote_endpoint(*iterator); - sock_.open(remote_endpoint.protocol()); if(bind_ip != "0.0.0.0" && bind_ip != "0" && bind_ip != "" ) { @@ -1119,7 +1241,7 @@ POP_WARNINGS MERROR("Error binding to " << bind_ip << ": " << ec.message()); if (sock_.is_open()) sock_.close(); - return false; + return CONNECT_FAILURE; } } @@ -1153,14 +1275,14 @@ POP_WARNINGS { if (sock_.is_open()) sock_.close(); - return false; + return CONNECT_FAILURE; } if(local_shared_context->ec == boost::asio::error::would_block && !r) { //timeout sock_.close(); _dbg3("Failed to connect to " << adr << ":" << port << ", because of timeout (" << conn_timeout << ")"); - return false; + return CONNECT_FAILURE; } } ec = local_shared_context->ec; @@ -1170,11 +1292,79 @@ POP_WARNINGS _dbg3("Some problems at connect, message: " << ec.message()); if (sock_.is_open()) sock_.close(); - return false; + return CONNECT_FAILURE; } _dbg3("Connected success to " << adr << ':' << port); + const epee::net_utils::ssl_support_t ssl_support = new_connection_l->get_ssl_support(); + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) + { + // Handshake + MDEBUG("Handshaking SSL..."); + if (!new_connection_l->handshake(boost::asio::ssl::stream_base::client)) + { + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) + { + boost::system::error_code ignored_ec; + sock_.shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec); + sock_.close(); + return CONNECT_NO_SSL; + } + MERROR("SSL handshake failed"); + if (sock_.is_open()) + sock_.close(); + return CONNECT_FAILURE; + } + } + + return CONNECT_SUCCESS; + + CATCH_ENTRY_L0("boosted_tcp_server<t_protocol_handler>::try_connect", CONNECT_FAILURE); + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> + bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip, epee::net_utils::ssl_support_t ssl_support) + { + TRY_ENTRY(); + + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, ssl_support, m_ssl_context) ); + connections_mutex.lock(); + connections_.insert(new_connection_l); + MDEBUG("connections_ size now " << connections_.size()); + connections_mutex.unlock(); + epee::misc_utils::auto_scope_leave_caller scope_exit_handler = epee::misc_utils::create_scope_leave_handler([&](){ CRITICAL_REGION_LOCAL(connections_mutex); connections_.erase(new_connection_l); }); + boost::asio::ip::tcp::socket& sock_ = new_connection_l->socket(); + + ////////////////////////////////////////////////////////////////////////// + boost::asio::ip::tcp::resolver resolver(io_service_); + boost::asio::ip::tcp::resolver::query query(boost::asio::ip::tcp::v4(), adr, port, boost::asio::ip::tcp::resolver::query::canonical_name); + boost::asio::ip::tcp::resolver::iterator iterator = resolver.resolve(query); + boost::asio::ip::tcp::resolver::iterator end; + if(iterator == end) + { + _erro("Failed to resolve " << adr); + return false; + } + ////////////////////////////////////////////////////////////////////////// + + + //boost::asio::ip::tcp::endpoint remote_endpoint(boost::asio::ip::address::from_string(addr.c_str()), port); + boost::asio::ip::tcp::endpoint remote_endpoint(*iterator); + + auto try_connect_result = try_connect(new_connection_l, adr, port, sock_, remote_endpoint, bind_ip, conn_timeout, ssl_support); + if (try_connect_result == CONNECT_FAILURE) + return false; + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect && try_connect_result == CONNECT_NO_SSL) + { + // we connected, but could not connect with SSL, try without + MERROR("SSL handshake failed on an autodetect connection, reconnecting without SSL"); + new_connection_l->disable_ssl(); + try_connect_result = try_connect(new_connection_l, adr, port, sock_, remote_endpoint, bind_ip, conn_timeout, epee::net_utils::ssl_support_t::e_ssl_support_disabled); + if (try_connect_result != CONNECT_SUCCESS) + return false; + } + // start adds the connection to the config object's list, so we don't need to have it locally anymore connections_mutex.lock(); connections_.erase(new_connection_l); @@ -1187,7 +1377,8 @@ POP_WARNINGS } else { - _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_state->sock_count); } new_connection_l->save_dbg_log(); @@ -1198,10 +1389,10 @@ POP_WARNINGS } //--------------------------------------------------------------------------------- template<class t_protocol_handler> template<class t_callback> - bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip) + bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip, epee::net_utils::ssl_support_t ssl_support) { TRY_ENTRY(); - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type, ssl_support, m_ssl_context) ); connections_mutex.lock(); connections_.insert(new_connection_l); MDEBUG("connections_ size now " << connections_.size()); diff --git a/contrib/epee/include/net/connection_basic.hpp b/contrib/epee/include/net/connection_basic.hpp index 9b6fc14a7..328f9afbf 100644 --- a/contrib/epee/include/net/connection_basic.hpp +++ b/contrib/epee/include/net/connection_basic.hpp @@ -47,14 +47,25 @@ #include <memory> #include <boost/asio.hpp> +#include <boost/asio/ssl.hpp> #include "net/net_utils_base.h" +#include "net/net_ssl.h" #include "syncobj.h" namespace epee { namespace net_utils { + struct socket_stats + { + socket_stats() + : sock_count(0), sock_number(0) + {} + + std::atomic<long> sock_count; + std::atomic<long> sock_number; + }; /************************************************************************/ /* */ @@ -72,6 +83,8 @@ class connection_basic_pimpl; // PIMPL for this class std::string to_string(t_connection_type type); class connection_basic { // not-templated base class for rapid developmet of some code parts + // beware of removing const, net_utils::connection is sketchily doing a cast to prevent storing ptr twice + const boost::shared_ptr<socket_stats> m_stats; public: std::unique_ptr< connection_basic_pimpl > mI; // my Implementation @@ -84,15 +97,57 @@ class connection_basic { // not-templated base class for rapid developmet of som /// Strand to ensure the connection's handlers are not called concurrently. boost::asio::io_service::strand strand_; /// Socket for the connection. - boost::asio::ip::tcp::socket socket_; + ssl_context_t &m_ssl_context; + ssl_support_t m_ssl_support; + boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket_; - std::atomic<long> &m_ref_sock_count; // reference to external counter of existing sockets that we will ++/-- public: // first counter is the ++/-- count of current sockets, the other socket_number is only-increasing ++ number generator - connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number); + connection_basic(boost::asio::ip::tcp::socket&& socket, boost::shared_ptr<socket_stats> stats, ssl_support_t ssl_support, ssl_context_t &ssl_context); + connection_basic(boost::asio::io_service &io_service, boost::shared_ptr<socket_stats> stats, ssl_support_t ssl_support, ssl_context_t &ssl_context); virtual ~connection_basic() noexcept(false); + //! \return `socket_stats` object passed in construction (ptr never changes). + socket_stats& get_stats() noexcept { return *m_stats; /* verified in constructor */ } + connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number, ssl_support_t ssl, ssl_context_t &ssl_context); + + boost::asio::ip::tcp::socket& socket() { return socket_.next_layer(); } + ssl_support_t get_ssl_support() const { return m_ssl_support; } + void disable_ssl() { m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; } + + bool handshake(boost::asio::ssl::stream_base::handshake_type type) + { + return ssl_handshake(socket_, type, m_ssl_context); + } + + template<typename MutableBufferSequence, typename ReadHandler> + void async_read_some(const MutableBufferSequence &buffers, ReadHandler &&handler) + { + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + socket_.async_read_some(buffers, std::forward<ReadHandler>(handler)); + else + socket().async_read_some(buffers, std::forward<ReadHandler>(handler)); + } + + template<typename ConstBufferSequence, typename WriteHandler> + void async_write_some(const ConstBufferSequence &buffers, WriteHandler &&handler) + { + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + socket_.async_write_some(buffers, std::forward<WriteHandler>(handler)); + else + socket().async_write_some(buffers, std::forward<WriteHandler>(handler)); + } + + template<typename ConstBufferSequence, typename WriteHandler> + void async_write(const ConstBufferSequence &buffers, WriteHandler &&handler) + { + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + boost::asio::async_write(socket_, buffers, std::forward<WriteHandler>(handler)); + else + boost::asio::async_write(socket(), buffers, std::forward<WriteHandler>(handler)); + } + // various handlers to be called from connection class: void do_send_handler_write(const void * ptr , size_t cb); void do_send_handler_write_from_queue(const boost::system::error_code& e, size_t cb , int q_len); // from handle_write, sending next part diff --git a/contrib/epee/include/net/enums.h b/contrib/epee/include/net/enums.h new file mode 100644 index 000000000..078a4b274 --- /dev/null +++ b/contrib/epee/include/net/enums.h @@ -0,0 +1,65 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +namespace epee +{ +namespace net_utils +{ + enum class address_type : std::uint8_t + { + // Do not change values, this will break serialization + invalid = 0, + ipv4 = 1, + ipv6 = 2, + i2p = 3, + tor = 4 + }; + + enum class zone : std::uint8_t + { + invalid = 0, + public_ = 1, // public is keyword + i2p = 2, + tor = 3 + }; + + // implementations in src/net_utils_base.cpp + + //! \return String name of zone or "invalid" on error. + const char* zone_to_string(zone value) noexcept; + + //! \return `zone` enum of `value` or `zone::invalid` on error. + zone zone_from_string(boost::string_ref value) noexcept; +} // net_utils +} // epee + diff --git a/contrib/epee/include/net/http_client.h b/contrib/epee/include/net/http_client.h index e01691794..34b3ac06c 100644 --- a/contrib/epee/include/net/http_client.h +++ b/contrib/epee/include/net/http_client.h @@ -275,7 +275,10 @@ namespace net_utils chunked_state m_chunked_state; std::string m_chunked_cache; critical_section m_lock; - bool m_ssl; + epee::net_utils::ssl_support_t m_ssl_support; + std::pair<std::string, std::string> m_ssl_private_key_and_certificate_path; + std::list<std::string> m_ssl_allowed_certificates; + bool m_ssl_allow_any_cert; public: explicit http_simple_client_template() @@ -293,35 +296,39 @@ namespace net_utils , m_chunked_state() , m_chunked_cache() , m_lock() - , m_ssl(false) + , m_ssl_support(epee::net_utils::ssl_support_t::e_ssl_support_autodetect) {} const std::string &get_host() const { return m_host_buff; }; const std::string &get_port() const { return m_port; }; - bool set_server(const std::string& address, boost::optional<login> user, bool ssl = false) + bool set_server(const std::string& address, boost::optional<login> user, epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, const std::pair<std::string, std::string> &private_key_and_certificate_path = {}, const std::list<std::string> &allowed_ssl_certificates = {}, bool allow_any_cert = false) { http::url_content parsed{}; const bool r = parse_url(address, parsed); CHECK_AND_ASSERT_MES(r, false, "failed to parse url: " << address); - set_server(std::move(parsed.host), std::to_string(parsed.port), std::move(user), ssl); + set_server(std::move(parsed.host), std::to_string(parsed.port), std::move(user), ssl_support, private_key_and_certificate_path, allowed_ssl_certificates, allow_any_cert); return true; } - void set_server(std::string host, std::string port, boost::optional<login> user, bool ssl = false) + void set_server(std::string host, std::string port, boost::optional<login> user, epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, const std::pair<std::string, std::string> &private_key_and_certificate_path = {}, const std::list<std::string> &allowed_ssl_certificates = {}, bool allow_any_cert = false) { CRITICAL_REGION_LOCAL(m_lock); disconnect(); m_host_buff = std::move(host); m_port = std::move(port); m_auth = user ? http_client_auth{std::move(*user)} : http_client_auth{}; - m_ssl = ssl; + m_ssl_support = ssl_support; + m_ssl_private_key_and_certificate_path = private_key_and_certificate_path; + m_ssl_allowed_certificates = allowed_ssl_certificates; + m_ssl_allow_any_cert = allow_any_cert; + m_net_client.set_ssl(m_ssl_support, m_ssl_private_key_and_certificate_path, m_ssl_allowed_certificates, m_ssl_allow_any_cert); } bool connect(std::chrono::milliseconds timeout) { CRITICAL_REGION_LOCAL(m_lock); - return m_net_client.connect(m_host_buff, m_port, timeout, m_ssl); + return m_net_client.connect(m_host_buff, m_port, timeout, "0.0.0.0"); } //--------------------------------------------------------------------------- bool disconnect() @@ -330,10 +337,10 @@ namespace net_utils return m_net_client.disconnect(); } //--------------------------------------------------------------------------- - bool is_connected() + bool is_connected(bool *ssl = NULL) { CRITICAL_REGION_LOCAL(m_lock); - return m_net_client.is_connected(); + return m_net_client.is_connected(ssl); } //--------------------------------------------------------------------------- virtual bool handle_target_data(std::string& piece_of_transfer) diff --git a/contrib/epee/include/net/http_server_impl_base.h b/contrib/epee/include/net/http_server_impl_base.h index 5669824c1..236067580 100644 --- a/contrib/epee/include/net/http_server_impl_base.h +++ b/contrib/epee/include/net/http_server_impl_base.h @@ -58,7 +58,10 @@ namespace epee bool init(std::function<void(size_t, uint8_t*)> rng, const std::string& bind_port = "0", const std::string& bind_ip = "0.0.0.0", std::vector<std::string> access_control_origins = std::vector<std::string>(), - boost::optional<net_utils::http::login> user = boost::none) + boost::optional<net_utils::http::login> user = boost::none, + epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, + const std::pair<std::string, std::string> &private_key_and_certificate_path = {}, + const std::list<std::string> &allowed_certificates = std::list<std::string>(), bool allow_any_cert = false) { //set self as callback handler @@ -75,7 +78,7 @@ namespace epee m_net_server.get_config_object().m_user = std::move(user); MGINFO("Binding on " << bind_ip << ":" << bind_port); - bool res = m_net_server.init_server(bind_port, bind_ip); + bool res = m_net_server.init_server(bind_port, bind_ip, ssl_support, private_key_and_certificate_path, allowed_certificates, allow_any_cert); if(!res) { LOG_ERROR("Failed to bind server"); diff --git a/contrib/epee/include/net/net_helper.h b/contrib/epee/include/net/net_helper.h index 94744ac21..5d9bb61cf 100644 --- a/contrib/epee/include/net/net_helper.h +++ b/contrib/epee/include/net/net_helper.h @@ -40,6 +40,7 @@ #include <boost/lambda/lambda.hpp> #include <boost/interprocess/detail/atomic.hpp> #include "net/net_utils_base.h" +#include "net/net_ssl.h" #include "misc_language.h" #undef MONERO_DEFAULT_LOG_CATEGORY @@ -57,6 +58,13 @@ namespace net_utils class blocked_mode_client { + enum try_connect_result_t + { + CONNECT_SUCCESS, + CONNECT_FAILURE, + CONNECT_NO_SSL, + }; + struct handler_obj @@ -84,9 +92,9 @@ namespace net_utils m_connected(false), m_deadline(m_io_service), m_shutdowned(0), - m_ssl(false), - m_ctx(boost::asio::ssl::context::sslv23), - m_ssl_socket(m_io_service,m_ctx) + m_ssl_support(epee::net_utils::ssl_support_t::e_ssl_support_autodetect), + m_ctx({boost::asio::ssl::context(boost::asio::ssl::context::sslv23), {}}), + m_ssl_socket(new boost::asio::ssl::stream<boost::asio::ip::tcp::socket>(m_io_service,m_ctx.context)) { @@ -110,28 +118,92 @@ namespace net_utils catch(...) { /* ignore */ } } + inline void set_ssl(epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, const std::pair<std::string, std::string> &private_key_and_certificate_path = {}, const std::list<std::string> &allowed_certificates = std::list<std::string>(), bool allow_any_cert = false) + { + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_disabled) + m_ctx = {boost::asio::ssl::context(boost::asio::ssl::context::sslv23), {}}; + else + m_ctx = create_ssl_context(private_key_and_certificate_path, allowed_certificates, allow_any_cert); + m_ssl_support = ssl_support; + } + inline - bool connect(const std::string& addr, int port, std::chrono::milliseconds timeout, bool ssl = false, const std::string& bind_ip = "0.0.0.0") + bool connect(const std::string& addr, int port, std::chrono::milliseconds timeout, const std::string& bind_ip = "0.0.0.0") { - return connect(addr, std::to_string(port), timeout, ssl, bind_ip); + return connect(addr, std::to_string(port), timeout, bind_ip); } inline - bool connect(const std::string& addr, const std::string& port, std::chrono::milliseconds timeout, bool ssl = false, const std::string& bind_ip = "0.0.0.0") + try_connect_result_t try_connect(const std::string& addr, const std::string& port, const boost::asio::ip::tcp::endpoint &remote_endpoint, std::chrono::milliseconds timeout, const std::string& bind_ip, epee::net_utils::ssl_support_t ssl_support) { - m_connected = false; - m_ssl = ssl; + m_ssl_socket->next_layer().open(remote_endpoint.protocol()); + if(bind_ip != "0.0.0.0" && bind_ip != "0" && bind_ip != "" ) + { + boost::asio::ip::tcp::endpoint local_endpoint(boost::asio::ip::address::from_string(addr.c_str()), 0); + m_ssl_socket->next_layer().bind(local_endpoint); + } + + + m_deadline.expires_from_now(timeout); + + boost::system::error_code ec = boost::asio::error::would_block; + + m_ssl_socket->next_layer().async_connect(remote_endpoint, boost::lambda::var(ec) = boost::lambda::_1); + while (ec == boost::asio::error::would_block) + { + m_io_service.run_one(); + } + + if (!ec && m_ssl_socket->next_layer().is_open()) + { + m_connected = true; + m_deadline.expires_at(std::chrono::steady_clock::time_point::max()); + // SSL Options + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled || ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) + { + if (!ssl_handshake(*m_ssl_socket, boost::asio::ssl::stream_base::client, m_ctx)) + { + if (ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) + { + boost::system::error_code ignored_ec; + m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ignored_ec); + m_ssl_socket->next_layer().close(); + m_connected = false; + return CONNECT_NO_SSL; + } + else + { + MWARNING("Failed to establish SSL connection"); + m_connected = false; + return CONNECT_FAILURE; + } + } + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_enabled; + } + return CONNECT_SUCCESS; + }else + { + MWARNING("Some problems at connect, message: " << ec.message()); + return CONNECT_FAILURE; + } + + } + + inline + bool connect(const std::string& addr, const std::string& port, std::chrono::milliseconds timeout, const std::string& bind_ip = "0.0.0.0") + { + m_connected = false; try { - m_ssl_socket.next_layer().close(); + m_ssl_socket->next_layer().close(); // Set SSL options // disable sslv2 - m_ctx.set_options(boost::asio::ssl::context::default_workarounds | boost::asio::ssl::context::no_sslv2); - m_ctx.set_default_verify_paths(); + m_ctx.context.set_options(boost::asio::ssl::context::default_workarounds | boost::asio::ssl::context::no_sslv2); + m_ctx.context.set_default_verify_paths(); + m_ssl_socket.reset(new boost::asio::ssl::stream<boost::asio::ip::tcp::socket>(m_io_service, m_ctx.context)); // Get a list of endpoints corresponding to the server name. - ////////////////////////////////////////////////////////////////////////// @@ -151,45 +223,20 @@ namespace net_utils //boost::asio::ip::tcp::endpoint remote_endpoint(boost::asio::ip::address::from_string(addr.c_str()), port); boost::asio::ip::tcp::endpoint remote_endpoint(*iterator); - - m_ssl_socket.next_layer().open(remote_endpoint.protocol()); - if(bind_ip != "0.0.0.0" && bind_ip != "0" && bind_ip != "" ) - { - boost::asio::ip::tcp::endpoint local_endpoint(boost::asio::ip::address::from_string(addr.c_str()), 0); - m_ssl_socket.next_layer().bind(local_endpoint); - } - - - m_deadline.expires_from_now(timeout); - - - boost::system::error_code ec = boost::asio::error::would_block; - - m_ssl_socket.next_layer().async_connect(remote_endpoint, boost::lambda::var(ec) = boost::lambda::_1); - while (ec == boost::asio::error::would_block) - { - m_io_service.run_one(); - } - - if (!ec && m_ssl_socket.next_layer().is_open()) + try_connect_result_t try_connect_result = try_connect(addr, port, remote_endpoint, timeout, bind_ip, m_ssl_support); + if (try_connect_result == CONNECT_FAILURE) + return false; + if (m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_autodetect) { - m_connected = true; - m_deadline.expires_at(std::chrono::steady_clock::time_point::max()); - // SSL Options - if(m_ssl) { - // Disable verification of host certificate - m_ssl_socket.set_verify_mode(boost::asio::ssl::verify_peer); - // Handshake - m_ssl_socket.next_layer().set_option(boost::asio::ip::tcp::no_delay(true)); - m_ssl_socket.handshake(boost::asio::ssl::stream_base::client); + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_enabled; + if (try_connect_result == CONNECT_NO_SSL) + { + MERROR("SSL handshake failed on an autodetect connection, reconnecting without SSL"); + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; + if (try_connect(addr, port, remote_endpoint, timeout, bind_ip, m_ssl_support) != CONNECT_SUCCESS) + return false; } - return true; - }else - { - MWARNING("Some problems at connect, message: " << ec.message()); - return false; } - } catch(const boost::system::system_error& er) { @@ -213,9 +260,9 @@ namespace net_utils if(m_connected) { m_connected = false; - if(m_ssl) + if(m_ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_disabled) shutdown_ssl(); - m_ssl_socket.next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both); + m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both); } } @@ -342,9 +389,13 @@ namespace net_utils return true; } - bool is_connected() + bool is_connected(bool *ssl = NULL) { - return m_connected && m_ssl_socket.next_layer().is_open(); + if (!m_connected || !m_ssl_socket->next_layer().is_open()) + return false; + if (ssl) + *ssl = m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled; + return true; } inline @@ -506,15 +557,15 @@ namespace net_utils { m_deadline.cancel(); boost::system::error_code ec; - if(m_ssl) + if(m_ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_disabled) shutdown_ssl(); - m_ssl_socket.next_layer().cancel(ec); + m_ssl_socket->next_layer().cancel(ec); if(ec) MDEBUG("Problems at cancel: " << ec.message()); - m_ssl_socket.next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ec); + m_ssl_socket->next_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ec); if(ec) MDEBUG("Problems at shutdown: " << ec.message()); - m_ssl_socket.next_layer().close(ec); + m_ssl_socket->next_layer().close(ec); if(ec) MDEBUG("Problems at close: " << ec.message()); boost::interprocess::ipcdetail::atomic_write32(&m_shutdowned, 1); @@ -533,7 +584,7 @@ namespace net_utils boost::asio::ip::tcp::socket& get_socket() { - return m_ssl_socket.next_layer(); + return m_ssl_socket->next_layer(); } private: @@ -550,7 +601,7 @@ namespace net_utils // connect(), read_line() or write_line() functions to return. LOG_PRINT_L3("Timed out socket"); m_connected = false; - m_ssl_socket.next_layer().close(); + m_ssl_socket->next_layer().close(); // There is no longer an active deadline. The expiry is set to positive // infinity so that the actor takes no action until a new deadline is set. @@ -565,7 +616,7 @@ namespace net_utils // ssl socket shutdown blocks if server doesn't respond. We close after 2 secs boost::system::error_code ec = boost::asio::error::would_block; m_deadline.expires_from_now(std::chrono::milliseconds(2000)); - m_ssl_socket.async_shutdown(boost::lambda::var(ec) = boost::lambda::_1); + m_ssl_socket->async_shutdown(boost::lambda::var(ec) = boost::lambda::_1); while (ec == boost::asio::error::would_block) { m_io_service.run_one(); @@ -586,35 +637,39 @@ namespace net_utils bool write(const void* data, size_t sz, boost::system::error_code& ec) { bool success; - if(m_ssl) - success = boost::asio::write(m_ssl_socket, boost::asio::buffer(data, sz), ec); + if(m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + success = boost::asio::write(*m_ssl_socket, boost::asio::buffer(data, sz), ec); else - success = boost::asio::write(m_ssl_socket.next_layer(), boost::asio::buffer(data, sz), ec); + success = boost::asio::write(m_ssl_socket->next_layer(), boost::asio::buffer(data, sz), ec); return success; } void async_write(const void* data, size_t sz, boost::system::error_code& ec) { - if(m_ssl) - boost::asio::async_write(m_ssl_socket, boost::asio::buffer(data, sz), boost::lambda::var(ec) = boost::lambda::_1); + if(m_ssl_support == epee::net_utils::ssl_support_t::e_ssl_support_enabled) + boost::asio::async_write(*m_ssl_socket, boost::asio::buffer(data, sz), boost::lambda::var(ec) = boost::lambda::_1); else - boost::asio::async_write(m_ssl_socket.next_layer(), boost::asio::buffer(data, sz), boost::lambda::var(ec) = boost::lambda::_1); + boost::asio::async_write(m_ssl_socket->next_layer(), boost::asio::buffer(data, sz), boost::lambda::var(ec) = boost::lambda::_1); } void async_read(char* buff, size_t sz, boost::asio::detail::transfer_at_least_t transfer_at_least, handler_obj& hndlr) { - if(!m_ssl) - boost::asio::async_read(m_ssl_socket.next_layer(), boost::asio::buffer(buff, sz), transfer_at_least, hndlr); + if(m_ssl_support != epee::net_utils::ssl_support_t::e_ssl_support_enabled) + boost::asio::async_read(m_ssl_socket->next_layer(), boost::asio::buffer(buff, sz), transfer_at_least, hndlr); else - boost::asio::async_read(m_ssl_socket, boost::asio::buffer(buff, sz), transfer_at_least, hndlr); + boost::asio::async_read(*m_ssl_socket, boost::asio::buffer(buff, sz), transfer_at_least, hndlr); } protected: boost::asio::io_service m_io_service; - boost::asio::ssl::context m_ctx; - boost::asio::ssl::stream<boost::asio::ip::tcp::socket> m_ssl_socket; - bool m_ssl; + epee::net_utils::ssl_context_t m_ctx; + std::shared_ptr<boost::asio::ssl::stream<boost::asio::ip::tcp::socket>> m_ssl_socket; + epee::net_utils::ssl_support_t m_ssl_support; + std::string m_ssl_private_key; + std::string m_ssl_certificate; + std::list<std::string> m_ssl_allowed_certificates; + bool m_ssl_allow_any_cerl; bool m_initialized; bool m_connected; boost::asio::steady_timer m_deadline; @@ -722,7 +777,7 @@ namespace net_utils // asynchronous operations are cancelled. This allows the blocked // connect(), read_line() or write_line() functions to return. LOG_PRINT_L3("Timed out socket"); - m_ssl_socket.next_layer().close(); + m_ssl_socket->next_layer().close(); // There is no longer an active deadline. The expiry is set to positive // infinity so that the actor takes no action until a new deadline is set. diff --git a/contrib/epee/include/net/net_ssl.h b/contrib/epee/include/net/net_ssl.h new file mode 100644 index 000000000..9ae1883af --- /dev/null +++ b/contrib/epee/include/net/net_ssl.h @@ -0,0 +1,68 @@ +// Copyright (c) 2006-2013, Andrey N. Sabelnikov, www.sabelnikov.net +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are met: +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// * Neither the name of the Andrey N. Sabelnikov nor the +// names of its contributors may be used to endorse or promote products +// derived from this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +// ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +// WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER BE LIABLE FOR ANY +// DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +// (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +// LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +// ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +// SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// + + + +#ifndef _NET_SSL_H +#define _NET_SSL_H + +#include <stdint.h> +#include <string> +#include <list> +#include <boost/utility/string_ref.hpp> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/ssl.hpp> + +namespace epee +{ +namespace net_utils +{ + enum class ssl_support_t: uint8_t { + e_ssl_support_disabled, + e_ssl_support_enabled, + e_ssl_support_autodetect, + }; + + struct ssl_context_t + { + boost::asio::ssl::context context; + std::list<std::string> allowed_certificates; + bool allow_any_cert; + }; + + // https://security.stackexchange.com/questions/34780/checking-client-hello-for-https-classification + constexpr size_t get_ssl_magic_size() { return 9; } + bool is_ssl(const unsigned char *data, size_t len); + ssl_context_t create_ssl_context(const std::pair<std::string, std::string> &private_key_and_certificate_path, std::list<std::string> allowed_certificates, bool allow_any_cert); + void use_ssl_certificate(ssl_context_t &ssl_context, const std::pair<std::string, std::string> &private_key_and_certificate_path); + bool create_ssl_certificate(std::string &pkey_buffer, std::string &cert_buffer); + bool is_certificate_allowed(boost::asio::ssl::verify_context &ctx, const std::list<std::string> &allowed_certificates); + bool ssl_handshake(boost::asio::ssl::stream<boost::asio::ip::tcp::socket> &socket, boost::asio::ssl::stream_base::handshake_type type, const epee::net_utils::ssl_context_t &ssl_context); + bool ssl_support_from_string(ssl_support_t &ssl, boost::string_ref s); +} +} + +#endif //_NET_SSL_H diff --git a/contrib/epee/include/net/net_utils_base.h b/contrib/epee/include/net/net_utils_base.h index a9e458626..7b5b07ef2 100644 --- a/contrib/epee/include/net/net_utils_base.h +++ b/contrib/epee/include/net/net_utils_base.h @@ -33,6 +33,7 @@ #include <boost/asio/io_service.hpp> #include <typeinfo> #include <type_traits> +#include "enums.h" #include "serialization/keyvalue_serialization.h" #include "misc_log_ex.h" @@ -43,6 +44,12 @@ #define MAKE_IP( a1, a2, a3, a4 ) (a1|(a2<<8)|(a3<<16)|(a4<<24)) #endif +namespace net +{ + class tor_address; + class i2p_address; +} + namespace epee { namespace net_utils @@ -53,6 +60,10 @@ namespace net_utils uint16_t m_port; public: + constexpr ipv4_network_address() noexcept + : ipv4_network_address(0, 0) + {} + constexpr ipv4_network_address(uint32_t ip, uint16_t port) noexcept : m_ip(ip), m_port(port) {} @@ -67,9 +78,10 @@ namespace net_utils std::string host_str() const; bool is_loopback() const; bool is_local() const; - static constexpr uint8_t get_type_id() noexcept { return ID; } + static constexpr address_type get_type_id() noexcept { return address_type::ipv4; } + static constexpr zone get_zone() noexcept { return zone::public_; } + static constexpr bool is_blockable() noexcept { return true; } - static const uint8_t ID = 1; BEGIN_KV_SERIALIZE_MAP() KV_SERIALIZE(m_ip) KV_SERIALIZE(m_port) @@ -103,7 +115,9 @@ namespace net_utils virtual std::string host_str() const = 0; virtual bool is_loopback() const = 0; virtual bool is_local() const = 0; - virtual uint8_t get_type_id() const = 0; + virtual address_type get_type_id() const = 0; + virtual zone get_zone() const = 0; + virtual bool is_blockable() const = 0; }; template<typename T> @@ -131,7 +145,9 @@ namespace net_utils virtual std::string host_str() const override { return value.host_str(); } virtual bool is_loopback() const override { return value.is_loopback(); } virtual bool is_local() const override { return value.is_local(); } - virtual uint8_t get_type_id() const override { return value.get_type_id(); } + virtual address_type get_type_id() const override { return value.get_type_id(); } + virtual zone get_zone() const override { return value.get_zone(); } + virtual bool is_blockable() const override { return value.is_blockable(); } }; std::shared_ptr<interface> self; @@ -146,6 +162,23 @@ namespace net_utils throw std::bad_cast{}; return static_cast<implementation<Type_>*>(self_)->value; } + + template<typename T, typename t_storage> + bool serialize_addr(std::false_type, t_storage& stg, typename t_storage::hsection hparent) + { + T addr{}; + if (!epee::serialization::selector<false>::serialize(addr, stg, hparent, "addr")) + return false; + *this = std::move(addr); + return true; + } + + template<typename T, typename t_storage> + bool serialize_addr(std::true_type, t_storage& stg, typename t_storage::hsection hparent) const + { + return epee::serialization::selector<true>::serialize(as<T>(), stg, hparent, "addr"); + } + public: network_address() : self(nullptr) {} template<typename T> @@ -158,43 +191,34 @@ namespace net_utils std::string host_str() const { return self ? self->host_str() : "<none>"; } bool is_loopback() const { return self ? self->is_loopback() : false; } bool is_local() const { return self ? self->is_local() : false; } - uint8_t get_type_id() const { return self ? self->get_type_id() : 0; } + address_type get_type_id() const { return self ? self->get_type_id() : address_type::invalid; } + zone get_zone() const { return self ? self->get_zone() : zone::invalid; } + bool is_blockable() const { return self ? self->is_blockable() : false; } template<typename Type> const Type &as() const { return as_mutable<const Type>(); } BEGIN_KV_SERIALIZE_MAP() - uint8_t type = is_store ? this_ref.get_type_id() : 0; + // need to `#include "net/[i2p|tor]_address.h"` when serializing `network_address` + static constexpr std::integral_constant<bool, is_store> is_store_{}; + + std::uint8_t type = std::uint8_t(is_store ? this_ref.get_type_id() : address_type::invalid); if (!epee::serialization::selector<is_store>::serialize(type, stg, hparent_section, "type")) return false; - switch (type) + + switch (address_type(type)) { - case ipv4_network_address::ID: - { - if (!is_store) - { - const_cast<network_address&>(this_ref) = ipv4_network_address{0, 0}; - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - MDEBUG("Found as addr: " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as<ipv4_network_address>()")) - MDEBUG("Found as template as<ipv4_network_address>(): " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as_mutable<ipv4_network_address>()")) - MDEBUG("Found as template as_mutable<ipv4_network_address>(): " << this_ref.str()); - else - { - MWARNING("Address not found"); - return false; - } - } - else - { - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (!epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - return false; - } + case address_type::ipv4: + return this_ref.template serialize_addr<ipv4_network_address>(is_store_, stg, hparent_section); + case address_type::tor: + return this_ref.template serialize_addr<net::tor_address>(is_store_, stg, hparent_section); + case address_type::i2p: + return this_ref.template serialize_addr<net::i2p_address>(is_store_, stg, hparent_section); + case address_type::invalid: + default: break; - } - default: MERROR("Unsupported network address type: " << (unsigned)type); return false; } + + MERROR("Unsupported network address type: " << (unsigned)type); + return false; END_KV_SERIALIZE_MAP() }; @@ -211,8 +235,6 @@ namespace net_utils inline bool operator>=(const network_address& lhs, const network_address& rhs) { return !lhs.less(rhs); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port = 0); - /************************************************************************/ /* */ /************************************************************************/ @@ -222,6 +244,7 @@ namespace net_utils const network_address m_remote_address; const bool m_is_income; const time_t m_started; + const time_t m_ssl; time_t m_last_recv; time_t m_last_send; uint64_t m_recv_cnt; @@ -232,13 +255,14 @@ namespace net_utils double m_max_speed_up; connection_context_base(boost::uuids::uuid connection_id, - const network_address &remote_address, bool is_income, + const network_address &remote_address, bool is_income, bool ssl, time_t last_recv = 0, time_t last_send = 0, uint64_t recv_cnt = 0, uint64_t send_cnt = 0): m_connection_id(connection_id), m_remote_address(remote_address), m_is_income(is_income), m_started(time(NULL)), + m_ssl(ssl), m_last_recv(last_recv), m_last_send(last_send), m_recv_cnt(recv_cnt), @@ -250,9 +274,10 @@ namespace net_utils {} connection_context_base(): m_connection_id(), - m_remote_address(ipv4_network_address{0,0}), + m_remote_address(), m_is_income(false), m_started(time(NULL)), + m_ssl(false), m_last_recv(0), m_last_send(0), m_recv_cnt(0), @@ -265,17 +290,17 @@ namespace net_utils connection_context_base& operator=(const connection_context_base& a) { - set_details(a.m_connection_id, a.m_remote_address, a.m_is_income); + set_details(a.m_connection_id, a.m_remote_address, a.m_is_income, a.m_ssl); return *this; } private: template<class t_protocol_handler> friend class connection; - void set_details(boost::uuids::uuid connection_id, const network_address &remote_address, bool is_income) + void set_details(boost::uuids::uuid connection_id, const network_address &remote_address, bool is_income, bool ssl) { this->~connection_context_base(); - new(this) connection_context_base(connection_id, remote_address, is_income); + new(this) connection_context_base(connection_id, remote_address, is_income, ssl); } }; diff --git a/contrib/epee/include/net/network_throttle-detail.hpp b/contrib/epee/include/net/network_throttle-detail.hpp index 955668d62..9d12291f4 100644 --- a/contrib/epee/include/net/network_throttle-detail.hpp +++ b/contrib/epee/include/net/network_throttle-detail.hpp @@ -36,6 +36,7 @@ #ifndef INCLUDED_throttle_detail_hpp #define INCLUDED_throttle_detail_hpp +#include <boost/circular_buffer.hpp> #include "network_throttle.hpp" namespace epee @@ -61,7 +62,7 @@ class network_throttle : public i_network_throttle { network_time_seconds m_slot_size; // the size of one slot. TODO: now hardcoded for 1 second e.g. in time_to_slot() // TODO for big window size, for performance better the substract on change of m_last_sample_time instead of recalculating average of eg >100 elements - std::vector< packet_info > m_history; // the history of bw usage + boost::circular_buffer< packet_info > m_history; // the history of bw usage network_time_seconds m_last_sample_time; // time of last history[0] - so we know when to rotate the buffer network_time_seconds m_start_time; // when we were created bool m_any_packet_yet; // did we yet got any packet to count diff --git a/contrib/epee/include/stats.h b/contrib/epee/include/stats.h new file mode 100644 index 000000000..1cf9c68fb --- /dev/null +++ b/contrib/epee/include/stats.h @@ -0,0 +1,58 @@ +#pragma once + +#include <vector> + +template<typename T, typename Tpod = T> +class Stats +{ +public: + Stats(const std::vector<T> &v): values(v), cached(0) {} + ~Stats() {} + + size_t get_size() const; + Tpod get_min() const; + Tpod get_max() const; + Tpod get_median() const; + double get_mean() const; + double get_confidence_interval_95() const; + double get_confidence_interval_99() const; + double get_standard_deviation() const; + double get_standard_error() const; + double get_variance() const; + double get_kurtosis() const; + double get_non_parametric_skew() const; + double get_t_test(T t) const; + double get_t_test(size_t npoints, double mean, double stddev) const; + double get_t_test(const Stats<T> &other) const; + double get_z_test(const Stats<T> &other) const; + double get_test(const Stats<T> &other) const; + std::vector<Tpod> get_quantiles(unsigned int quantiles) const; + std::vector<size_t> get_bins(unsigned int bins) const; + bool is_same_distribution_95(size_t npoints, double mean, double stddev) const; + bool is_same_distribution_95(const Stats<T> &other) const; + bool is_same_distribution_99(size_t npoints, double mean, double stddev) const; + bool is_same_distribution_99(const Stats<T> &other) const; + + double get_cdf95(size_t df) const; + double get_cdf95(const Stats<T> &other) const; + double get_cdf99(size_t df) const; + double get_cdf99(const Stats<T> &other) const; + +private: + inline bool is_cached(int bit) const; + inline void set_cached(int bit) const; + + const std::vector<T> &values; + + mutable uint64_t cached; + mutable Tpod min; + mutable Tpod max; + mutable Tpod median; + mutable double mean; + mutable double standard_deviation; + mutable double standard_error; + mutable double variance; + mutable double kurtosis; +}; + +#include "stats.inl" diff --git a/contrib/epee/include/stats.inl b/contrib/epee/include/stats.inl new file mode 100644 index 000000000..5a5cd0b93 --- /dev/null +++ b/contrib/epee/include/stats.inl @@ -0,0 +1,359 @@ +#include <math.h> +#include <limits> +#include <algorithm> +#include "stats.h" + +enum +{ + bit_min = 0, + bit_max, + bit_median, + bit_mean, + bit_standard_deviation, + bit_standard_error, + bit_variance, + bit_kurtosis, +}; + +static inline double square(double x) +{ + return x * x; +} + +template<typename T> +static inline double interpolate(T v, T v0, double i0, T v1, double i1) +{ + return i0 + (i1 - i0) * (v - v0) / (v1 - v0); +} + +template<typename T, typename Tpod> +inline bool Stats<T, Tpod>::is_cached(int bit) const +{ + return cached & (1<<bit); +} + +template<typename T, typename Tpod> +inline void Stats<T, Tpod>::set_cached(int bit) const +{ + cached |= 1<<bit; +} + +template<typename T, typename Tpod> +size_t Stats<T, Tpod>::get_size() const +{ + return values.size(); +} + +template<typename T, typename Tpod> +Tpod Stats<T, Tpod>::get_min() const +{ + if (!is_cached(bit_min)) + { + min = std::numeric_limits<Tpod>::max(); + for (const T &v: values) + min = std::min<Tpod>(min, v); + set_cached(bit_min); + } + return min; +} + +template<typename T, typename Tpod> +Tpod Stats<T, Tpod>::get_max() const +{ + if (!is_cached(bit_max)) + { + max = std::numeric_limits<Tpod>::min(); + for (const T &v: values) + max = std::max<Tpod>(max, v); + set_cached(bit_max); + } + return max; +} + +template<typename T, typename Tpod> +Tpod Stats<T, Tpod>::get_median() const +{ + if (!is_cached(bit_median)) + { + std::vector<Tpod> sorted; + sorted.reserve(values.size()); + for (const T &v: values) + sorted.push_back(v); + std::sort(sorted.begin(), sorted.end()); + if (sorted.size() & 1) + { + median = sorted[sorted.size() / 2]; + } + else + { + median = (sorted[(sorted.size() - 1) / 2] + sorted[sorted.size() / 2]) / 2; + } + set_cached(bit_median); + } + return median; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_mean() const +{ + if (values.empty()) + return 0.0; + if (!is_cached(bit_mean)) + { + mean = 0.0; + for (const T &v: values) + mean += v; + mean /= values.size(); + set_cached(bit_mean); + } + return mean; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_cdf95(size_t df) const +{ + static const double p[101] = { + -1, 12.706, 4.3027, 3.1824, 2.7765, 2.5706, 2.4469, 2.3646, 2.3060, 2.2622, 2.2281, 2.2010, 2.1788, 2.1604, 2.1448, 2.1315, + 2.1199, 2.1098, 2.1009, 2.0930, 2.0860, 2.0796, 2.0739, 2.0687, 2.0639, 2.0595, 2.0555, 2.0518, 2.0484, 2.0452, 2.0423, 2.0395, + 2.0369, 2.0345, 2.0322, 2.0301, 2.0281, 2.0262, 2.0244, 2.0227, 2.0211, 2.0195, 2.0181, 2.0167, 2.0154, 2.0141, 2.0129, 2.0117, + 2.0106, 2.0096, 2.0086, 2.0076, 2.0066, 2.0057, 2.0049, 2.0040, 2.0032, 2.0025, 2.0017, 2.0010, 2.0003, 1.9996, 1.9990, 1.9983, + 1.9977, 1.9971, 1.9966, 1.9960, 1.9955, 1.9949, 1.9944, 1.9939, 1.9935, 1.9930, 1.9925, 1.9921, 1.9917, 1.9913, 1.9908, 1.9905, + 1.9901, 1.9897, 1.9893, 1.9890, 1.9886, 1.9883, 1.9879, 1.9876, 1.9873, 1.9870, 1.9867, 1.9864, 1.9861, 1.9858, 1.9855, 1.9852, + 1.9850, 1.9847, 1.9845, 1.9842, 1.9840, + }; + if (df <= 100) + return p[df]; + if (df <= 120) + return interpolate<size_t>(df, 100, 1.9840, 120, 1.98); + return 1.96; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_cdf95(const Stats<T> &other) const +{ + return get_cdf95(get_size() + other.get_size() - 2); +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_cdf99(size_t df) const +{ + static const double p[101] = { + -1, 9.9250, 5.8408, 4.6041, 4.0321, 3.7074, 3.4995, 3.3554, 3.2498, 3.1693, 3.1058, 3.0545, 3.0123, 2.9768, 2.9467, 2.9208, 2.8982, + 2.8784, 2.8609, 2.8453, 2.8314, 2.8188, 2.8073, 2.7970, 2.7874, 2.7787, 2.7707, 2.7633, 2.7564, 2.7500, 2.7440, 2.7385, 2.7333, + 2.7284, 2.7238, 2.7195, 2.7154, 2.7116, 2.7079, 2.7045, 2.7012, 2.6981, 2.6951, 2.6923, 2.6896, 2.6870, 2.6846, 2.6822, 2.6800, + 2.6778, 2.6757, 2.6737, 2.6718, 2.6700, 2.6682, 2.6665, 2.6649, 2.6633, 2.6618, 2.6603, 2.6589, 2.6575, 2.6561, 2.6549, 2.6536, + 2.6524, 2.6512, 2.6501, 2.6490, 2.6479, 2.6469, 2.6458, 2.6449, 2.6439, 2.6430, 2.6421, 2.6412, 2.6403, 2.6395, 2.6387, 2.6379, + 2.6371, 2.6364, 2.6356, 2.6349, 2.6342, 2.6335, 2.6329, 2.6322, 2.6316, 2.6309, 2.6303, 2.6297, 2.6291, 2.6286, 2.6280, 2.6275, + 2.6269, 2.6264, 2.6259, + }; + if (df <= 100) + return p[df]; + if (df <= 120) + return interpolate<size_t>(df, 100, 2.6529, 120, 2.617); + return 2.576; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_cdf99(const Stats<T> &other) const +{ + return get_cdf99(get_size() + other.get_size() - 2); +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_confidence_interval_95() const +{ + const size_t df = get_size() - 1; + return get_standard_error() * get_cdf95(df); +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_confidence_interval_99() const +{ + const size_t df = get_size() - 1; + return get_standard_error() * get_cdf99(df); +} + +template<typename T, typename Tpod> +bool Stats<T, Tpod>::is_same_distribution_95(size_t npoints, double mean, double stddev) const +{ + return fabs(get_t_test(npoints, mean, stddev)) < get_cdf95(get_size() + npoints - 2); +} + +template<typename T, typename Tpod> +bool Stats<T, Tpod>::is_same_distribution_95(const Stats<T> &other) const +{ + return fabs(get_t_test(other)) < get_cdf95(other); +} + +template<typename T, typename Tpod> +bool Stats<T, Tpod>::is_same_distribution_99(size_t npoints, double mean, double stddev) const +{ + return fabs(get_t_test(npoints, mean, stddev)) < get_cdf99(get_size() + npoints - 2); +} + +template<typename T, typename Tpod> +bool Stats<T, Tpod>::is_same_distribution_99(const Stats<T> &other) const +{ + return fabs(get_t_test(other)) < get_cdf99(other); +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_standard_deviation() const +{ + if (values.size() <= 1) + return 0.0; + if (!is_cached(bit_standard_deviation)) + { + Tpod m = get_mean(), t = 0; + for (const T &v: values) + t += ((T)v - m) * ((T)v - m); + standard_deviation = sqrt(t / ((double)values.size() - 1)); + set_cached(bit_standard_deviation); + } + return standard_deviation; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_standard_error() const +{ + if (!is_cached(bit_standard_error)) + { + standard_error = get_standard_deviation() / sqrt(get_size()); + set_cached(bit_standard_error); + } + return standard_error; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_variance() const +{ + if (!is_cached(bit_variance)) + { + double stddev = get_standard_deviation(); + variance = stddev * stddev; + set_cached(bit_variance); + } + return variance; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_kurtosis() const +{ + if (values.empty()) + return 0.0; + if (!is_cached(bit_kurtosis)) + { + double m = get_mean(); + double n = 0, d = 0; + for (const T &v: values) + { + T p2 = (v - m) * (v - m); + T p4 = p2 * p2; + n += p4; + d += p2; + } + n /= values.size(); + d /= values.size(); + d *= d; + kurtosis = n / d; + set_cached(bit_kurtosis); + } + return kurtosis; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_non_parametric_skew() const +{ + return (get_mean() - get_median()) / get_standard_deviation(); +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_t_test(T t) const +{ + const double n = get_mean() - t; + const double d = get_standard_deviation() / sqrt(get_size()); + return n / d; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_t_test(size_t npoints, double mean, double stddev) const +{ + const double n = get_mean() - mean; + const double d = sqrt(get_variance() / get_size() + square(stddev) / npoints); + return n / d; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_t_test(const Stats<T> &other) const +{ + const double n = get_mean() - other.get_mean(); + const double d = sqrt(get_variance() / get_size() + other.get_variance() / other.get_size()); + return n / d; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_z_test(const Stats<T> &other) const +{ + const double m0 = get_mean(); + const double m1 = other.get_mean(); + const double sd0 = get_standard_deviation(); + const double sd1 = other.get_standard_deviation(); + const size_t s0 = get_size(); + const size_t s1 = other.get_size(); + + const double n = m0 - m1; + const double d = sqrt(square(sd0 / sqrt(s0)) + square(sd1 / sqrt(s1))); + + return n / d; +} + +template<typename T, typename Tpod> +double Stats<T, Tpod>::get_test(const Stats<T> &other) const +{ + if (get_size() >= 30 && other.get_size() >= 30) + return get_z_test(other); + else + return get_t_test(other); +} + +template<typename T, typename Tpod> +std::vector<Tpod> Stats<T, Tpod>::get_quantiles(unsigned int quantiles) const +{ + std::vector<Tpod> sorted; + sorted.reserve(values.size()); + for (const T &v: values) + sorted.push_back(v); + std::sort(sorted.begin(), sorted.end()); + std::vector<Tpod> q(quantiles + 1, 0); + for (unsigned int i = 1; i <= quantiles; ++i) + { + unsigned idx = (unsigned)ceil(values.size() * i / (double)quantiles); + q[i] = sorted[idx - 1]; + } + if (!is_cached(bit_min)) + { + min = sorted.front(); + set_cached(bit_min); + } + q[0] = min; + if (!is_cached(bit_max)) + { + max = sorted.back(); + set_cached(bit_max); + } + return q; +} + +template<typename T, typename Tpod> +std::vector<size_t> Stats<T, Tpod>::get_bins(unsigned int bins) const +{ + std::vector<size_t> b(bins, 0); + const double scale = 1.0 / (get_max() - get_min()); + const T base = get_min(); + for (const T &v: values) + { + unsigned int idx = (v - base) * scale; + ++b[idx]; + } + return b; +} diff --git a/contrib/epee/src/CMakeLists.txt b/contrib/epee/src/CMakeLists.txt index cea50c9dd..e913211ea 100644 --- a/contrib/epee/src/CMakeLists.txt +++ b/contrib/epee/src/CMakeLists.txt @@ -27,7 +27,7 @@ # THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. add_library(epee STATIC hex.cpp http_auth.cpp mlog.cpp net_utils_base.cpp string_tools.cpp wipeable_string.cpp memwipe.c - connection_basic.cpp network_throttle.cpp network_throttle-detail.cpp mlocker.cpp buffer.cpp) + connection_basic.cpp network_throttle.cpp network_throttle-detail.cpp mlocker.cpp buffer.cpp net_ssl.cpp) if (USE_READLINE AND GNU_READLINE_FOUND) add_library(epee_readline STATIC readline_buffer.cpp) endif() diff --git a/contrib/epee/src/connection_basic.cpp b/contrib/epee/src/connection_basic.cpp index 7d145ee46..377fb3452 100644 --- a/contrib/epee/src/connection_basic.cpp +++ b/contrib/epee/src/connection_basic.cpp @@ -48,7 +48,7 @@ #include "net/network_throttle-detail.hpp" #undef MONERO_DEFAULT_LOG_CATEGORY -#define MONERO_DEFAULT_LOG_CATEGORY "net.p2p" +#define MONERO_DEFAULT_LOG_CATEGORY "net.conn" // ################################################################################################ // local (TU local) headers @@ -103,7 +103,7 @@ namespace net_utils // connection_basic_pimpl // ================================================================================================ -connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name) { } +connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name), m_peer_number(0) { } // ================================================================================================ // connection_basic @@ -113,29 +113,61 @@ connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_thro int connection_basic_pimpl::m_default_tos; // methods: -connection_basic::connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number) - : +connection_basic::connection_basic(boost::asio::ip::tcp::socket&& sock, boost::shared_ptr<socket_stats> stats, ssl_support_t ssl_support, ssl_context_t &ssl_context) + : + m_stats(std::move(stats)), + mI( new connection_basic_pimpl("peer") ), + strand_(sock.get_io_service()), + socket_(sock.get_io_service(), ssl_context.context), + m_want_close_connection(false), + m_was_shutdown(false), + m_ssl_support(ssl_support), + m_ssl_context(ssl_context) +{ + // add nullptr checks if removed + CHECK_AND_ASSERT_THROW_MES(bool(m_stats), "stats shared_ptr cannot be null"); + + socket_.next_layer() = std::move(sock); + + ++(m_stats->sock_count); // increase the global counter + mI->m_peer_number = m_stats->sock_number.fetch_add(1); // use, and increase the generated number + + std::string remote_addr_str = "?"; + try { boost::system::error_code e; remote_addr_str = socket().remote_endpoint(e).address().to_string(); } catch(...){} ; + + _note("Spawned connection #"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_stats->sock_count); +} + +connection_basic::connection_basic(boost::asio::io_service &io_service, boost::shared_ptr<socket_stats> stats, ssl_support_t ssl_support, ssl_context_t &ssl_context) + : + m_stats(std::move(stats)), mI( new connection_basic_pimpl("peer") ), strand_(io_service), - socket_(io_service), + socket_(io_service, ssl_context.context), m_want_close_connection(false), m_was_shutdown(false), - m_ref_sock_count(ref_sock_count) -{ - ++ref_sock_count; // increase the global counter - mI->m_peer_number = sock_number.fetch_add(1); // use, and increase the generated number + m_ssl_support(ssl_support), + m_ssl_context(ssl_context) +{ + // add nullptr checks if removed + CHECK_AND_ASSERT_THROW_MES(bool(m_stats), "stats shared_ptr cannot be null"); + + ++(m_stats->sock_count); // increase the global counter + mI->m_peer_number = m_stats->sock_number.fetch_add(1); // use, and increase the generated number std::string remote_addr_str = "?"; - try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; + try { boost::system::error_code e; remote_addr_str = socket().remote_endpoint(e).address().to_string(); } catch(...){} ; - _note("Spawned connection p2p#"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_ref_sock_count); + _note("Spawned connection #"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_stats->sock_count); } connection_basic::~connection_basic() noexcept(false) { + --(m_stats->sock_count); + std::string remote_addr_str = "?"; - m_ref_sock_count--; - try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; - _note("Destructing connection p2p#"<<mI->m_peer_number << " to " << remote_addr_str); + try { boost::system::error_code e; remote_addr_str = socket().remote_endpoint(e).address().to_string(); } catch(...){} ; + _note("Destructing connection #"<<mI->m_peer_number << " to " << remote_addr_str); +try { throw 0; } catch(...){} } void connection_basic::set_rate_up_limit(uint64_t limit) { diff --git a/contrib/epee/src/net_ssl.cpp b/contrib/epee/src/net_ssl.cpp new file mode 100644 index 000000000..941799078 --- /dev/null +++ b/contrib/epee/src/net_ssl.cpp @@ -0,0 +1,319 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include <string.h> +#include <boost/asio/ssl.hpp> +#include <openssl/ssl.h> +#include <openssl/pem.h> +#include "misc_log_ex.h" +#include "net/net_ssl.h" + +#undef MONERO_DEFAULT_LOG_CATEGORY +#define MONERO_DEFAULT_LOG_CATEGORY "net.ssl" + +// openssl genrsa -out /tmp/KEY 4096 +// openssl req -new -key /tmp/KEY -out /tmp/REQ +// openssl x509 -req -days 999999 -sha256 -in /tmp/REQ -signkey /tmp/KEY -out /tmp/CERT + +namespace +{ + struct openssl_bio_free + { + void operator()(BIO* ptr) const noexcept + { + if (ptr) + BIO_free(ptr); + } + }; + using openssl_bio = std::unique_ptr<BIO, openssl_bio_free>; + + struct openssl_pkey_free + { + void operator()(EVP_PKEY* ptr) const noexcept + { + if (ptr) + EVP_PKEY_free(ptr); + } + }; + using openssl_pkey = std::unique_ptr<EVP_PKEY, openssl_pkey_free>; + +} + +namespace epee +{ +namespace net_utils +{ + +// https://stackoverflow.com/questions/256405/programmatically-create-x509-certificate-using-openssl +bool create_ssl_certificate(EVP_PKEY *&pkey, X509 *&cert) +{ + MGINFO("Generating SSL certificate"); + pkey = EVP_PKEY_new(); + openssl_pkey pkey_deleter{pkey}; + if (!pkey) + { + MERROR("Failed to create new private key"); + return false; + } + RSA *rsa = RSA_generate_key(4096, RSA_F4, NULL, NULL); + if (!rsa) + { + MERROR("Error generating RSA private key"); + return false; + } + if (EVP_PKEY_assign_RSA(pkey, rsa) <= 0) + { + RSA_free(rsa); + MERROR("Error assigning RSA private key"); + return false; + } + + cert = X509_new(); + if (!cert) + { + MERROR("Failed to create new X509 certificate"); + return false; + } + ASN1_INTEGER_set(X509_get_serialNumber(cert), 1); + X509_gmtime_adj(X509_get_notBefore(cert), 0); + X509_gmtime_adj(X509_get_notAfter(cert), 3600 * 24 * 182); // half a year + if (!X509_set_pubkey(cert, pkey)) + { + MERROR("Error setting pubkey on certificate"); + X509_free(cert); + return false; + } + X509_NAME *name = X509_get_subject_name(cert); + X509_set_issuer_name(cert, name); + + if (X509_sign(cert, pkey, EVP_sha256()) == 0) + { + MERROR("Error signing certificate"); + X509_free(cert); + return false; + } + return true; +} + +bool create_ssl_certificate(std::string &pkey_buffer, std::string &cert_buffer) +{ + EVP_PKEY *pkey; + X509 *cert; + if (!create_ssl_certificate(pkey, cert)) + return false; + BIO *bio_pkey = BIO_new(BIO_s_mem()), *bio_cert = BIO_new(BIO_s_mem()); + openssl_bio bio_pkey_deleter{bio_pkey}; + bool success = PEM_write_bio_PrivateKey(bio_pkey, pkey, NULL, NULL, 0, NULL, NULL) && PEM_write_bio_X509(bio_cert, cert); + X509_free(cert); + if (!success) + { + MERROR("Failed to write cert and/or pkey: " << ERR_get_error()); + return false; + } + BUF_MEM *buf = NULL; + BIO_get_mem_ptr(bio_pkey, &buf); + if (!buf || !buf->data || !buf->length) + { + MERROR("Failed to write pkey: " << ERR_get_error()); + return false; + } + pkey_buffer = std::string(buf->data, buf->length); + buf = NULL; + BIO_get_mem_ptr(bio_cert, &buf); + if (!buf || !buf->data || !buf->length) + { + MERROR("Failed to write cert: " << ERR_get_error()); + return false; + } + cert_buffer = std::string(buf->data, buf->length); + return success; +} + +ssl_context_t create_ssl_context(const std::pair<std::string, std::string> &private_key_and_certificate_path, std::list<std::string> allowed_certificates, bool allow_any_cert) +{ + ssl_context_t ssl_context({boost::asio::ssl::context(boost::asio::ssl::context::sslv23), std::move(allowed_certificates)}); + + // disable sslv2 + ssl_context.context.set_options(boost::asio::ssl::context::default_workarounds | boost::asio::ssl::context::no_sslv2); + ssl_context.context.set_default_verify_paths(); + + // set options on the SSL context for added security + SSL_CTX *ctx = ssl_context.context.native_handle(); + CHECK_AND_ASSERT_THROW_MES(ctx, "Failed to get SSL context"); + SSL_CTX_clear_options(ctx, SSL_OP_LEGACY_SERVER_CONNECT); // SSL_CTX_SET_OPTIONS(3) + SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_OFF); // https://stackoverflow.com/questions/22378442 +#ifdef SSL_OP_NO_TICKET + SSL_CTX_set_options(ctx, SSL_OP_NO_TICKET); // https://stackoverflow.com/questions/22378442 +#endif +#ifdef SSL_OP_NO_RENEGOTIATION + SSL_CTX_set_options(ctx, SSL_OP_NO_RENEGOTIATION); +#endif +#ifdef SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION + SSL_CTX_set_options(ctx, SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION); +#endif +#ifdef SSL_OP_NO_COMPRESSION + SSL_CTX_set_options(ctx, SSL_OP_NO_COMPRESSION); +#endif + SSL_CTX_set_options(ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_TLSv1_1); // https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices + + CHECK_AND_ASSERT_THROW_MES(private_key_and_certificate_path.first.empty() == private_key_and_certificate_path.second.empty(), "private key and certificate must be either both given or both empty"); + if (private_key_and_certificate_path.second.empty()) + { + std::string pkey, cert; + CHECK_AND_ASSERT_THROW_MES(create_ssl_certificate(pkey, cert), "Failed to create certificate"); + ssl_context.context.use_private_key(boost::asio::buffer(pkey), boost::asio::ssl::context::pem); + ssl_context.context.use_certificate(boost::asio::buffer(cert), boost::asio::ssl::context::pem); + } + else + { + ssl_context.context.use_private_key_file(private_key_and_certificate_path.first, boost::asio::ssl::context::pem); + ssl_context.context.use_certificate_file(private_key_and_certificate_path.second, boost::asio::ssl::context::pem); + } + ssl_context.allow_any_cert = allow_any_cert; + + return ssl_context; +} + +void use_ssl_certificate(ssl_context_t &ssl_context, const std::pair<std::string, std::string> &private_key_and_certificate_path) +{ + ssl_context.context.use_private_key_file(private_key_and_certificate_path.first, boost::asio::ssl::context::pem); + ssl_context.context.use_certificate_file(private_key_and_certificate_path.second, boost::asio::ssl::context::pem); +} + +bool is_ssl(const unsigned char *data, size_t len) +{ + if (len < get_ssl_magic_size()) + return false; + + // https://security.stackexchange.com/questions/34780/checking-client-hello-for-https-classification + MDEBUG("SSL detection buffer, " << len << " bytes: " + << (unsigned)(unsigned char)data[0] << " " << (unsigned)(unsigned char)data[1] << " " + << (unsigned)(unsigned char)data[2] << " " << (unsigned)(unsigned char)data[3] << " " + << (unsigned)(unsigned char)data[4] << " " << (unsigned)(unsigned char)data[5] << " " + << (unsigned)(unsigned char)data[6] << " " << (unsigned)(unsigned char)data[7] << " " + << (unsigned)(unsigned char)data[8]); + if (data[0] == 0x16) // record + if (data[1] == 3) // major version + if (data[5] == 1) // ClientHello + if (data[6] == 0 && data[3]*256 + data[4] == data[7]*256 + data[8] + 4) // length check + return true; + return false; +} + +bool is_certificate_allowed(boost::asio::ssl::verify_context &ctx, const std::list<std::string> &allowed_certificates) +{ + X509_STORE_CTX *sctx = ctx.native_handle(); + if (!sctx) + { + MERROR("Error getting verify_context handle"); + return false; + } + X509 *cert =X509_STORE_CTX_get_current_cert(sctx); + if (!cert) + { + MERROR("No certificate found in verify_context"); + return false; + } + + BIO *bio_cert = BIO_new(BIO_s_mem()); + openssl_bio bio_cert_deleter{bio_cert}; + bool success = PEM_write_bio_X509(bio_cert, cert); + if (!success) + { + MERROR("Failed to print certificate"); + return false; + } + BUF_MEM *buf = NULL; + BIO_get_mem_ptr(bio_cert, &buf); + if (!buf || !buf->data || !buf->length) + { + MERROR("Failed to write certificate: " << ERR_get_error()); + return false; + } + std::string certificate(std::string(buf->data, buf->length)); + return std::find(allowed_certificates.begin(), allowed_certificates.end(), certificate) != allowed_certificates.end(); +} + +bool ssl_handshake(boost::asio::ssl::stream<boost::asio::ip::tcp::socket> &socket, boost::asio::ssl::stream_base::handshake_type type, const epee::net_utils::ssl_context_t &ssl_context) +{ + bool verified = false; + socket.next_layer().set_option(boost::asio::ip::tcp::no_delay(true)); + socket.set_verify_mode(boost::asio::ssl::verify_peer); + socket.set_verify_callback([&](bool preverified, boost::asio::ssl::verify_context &ctx) + { + if (!preverified) + { + const int err = X509_STORE_CTX_get_error(ctx.native_handle()); + const int depth = X509_STORE_CTX_get_error_depth(ctx.native_handle()); + if (err != X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT || depth != 0) + { + MERROR("Invalid SSL certificate, error " << err << " at depth " << depth << ", connection dropped"); + return false; + } + } + if (!ssl_context.allow_any_cert && !ssl_context.allowed_certificates.empty() && !is_certificate_allowed(ctx, ssl_context.allowed_certificates)) + { + MERROR("Certificate is not in the allowed list, connection droppped"); + return false; + } + verified = true; + return true; + }); + + boost::system::error_code ec; + socket.handshake(type, ec); + if (ec) + { + MERROR("handshake failed, connection dropped"); + return false; + } + if (!ssl_context.allow_any_cert && !verified) + { + MERROR("Peer did not provide a certificate in the allowed list, connection dropped"); + return false; + } + MDEBUG("SSL handshake success"); + return true; +} + +bool ssl_support_from_string(ssl_support_t &ssl, boost::string_ref s) +{ + if (s == "enabled") + ssl = epee::net_utils::ssl_support_t::e_ssl_support_enabled; + else if (s == "disabled") + ssl = epee::net_utils::ssl_support_t::e_ssl_support_disabled; + else if (s == "autodetect") + ssl = epee::net_utils::ssl_support_t::e_ssl_support_autodetect; + else + return false; + return true; +} + +} // namespace +} // namespace + diff --git a/contrib/epee/src/net_utils_base.cpp b/contrib/epee/src/net_utils_base.cpp index 263b344b4..9b781027e 100644 --- a/contrib/epee/src/net_utils_base.cpp +++ b/contrib/epee/src/net_utils_base.cpp @@ -8,8 +8,6 @@ namespace epee { namespace net_utils { - const uint8_t ipv4_network_address::ID; - bool ipv4_network_address::equal(const ipv4_network_address& other) const noexcept { return is_same_host(other) && port() == other.port(); } @@ -58,20 +56,6 @@ namespace epee { namespace net_utils return self_->is_same_host(*other_self); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port) - { - uint32_t ip; - uint16_t port; - if (epee::string_tools::parse_peer_from_string(ip, port, string)) - { - if (default_port && !port) - port = default_port; - address = ipv4_network_address{ip, port}; - return true; - } - return false; - } - std::string print_connection_context(const connection_context_base& ctx) { std::stringstream ss; @@ -86,5 +70,31 @@ namespace epee { namespace net_utils return ss.str(); } + const char* zone_to_string(zone value) noexcept + { + switch (value) + { + case zone::public_: + return "public"; + case zone::i2p: + return "i2p"; + case zone::tor: + return "tor"; + default: + break; + } + return "invalid"; + } + + zone zone_from_string(const boost::string_ref value) noexcept + { + if (value == "public") + return zone::public_; + if (value == "i2p") + return zone::i2p; + if (value == "tor") + return zone::tor; + return zone::invalid; + } }} diff --git a/contrib/epee/src/network_throttle-detail.cpp b/contrib/epee/src/network_throttle-detail.cpp index d2e776df0..0b42402bd 100644 --- a/contrib/epee/src/network_throttle-detail.cpp +++ b/contrib/epee/src/network_throttle-detail.cpp @@ -135,6 +135,7 @@ network_throttle::network_throttle(const std::string &nameshort, const std::stri m_slot_size = 1.0; // hard coded in few places m_target_speed = 16 * 1024; // other defaults are probably defined in the command-line parsing code when this class is used e.g. as main global throttle m_last_sample_time = 0; + m_history.resize(m_window_size); } void network_throttle::set_name(const std::string &name) @@ -168,8 +169,7 @@ void network_throttle::tick() { _dbg3("Moving counter buffer by 1 second " << last_sample_time_slot << " < " << current_sample_time_slot << " (last time " << m_last_sample_time<<")"); // rotate buffer - for (size_t i=m_history.size()-1; i>=1; --i) m_history[i] = m_history[i-1]; - m_history[0] = packet_info(); + m_history.push_front(packet_info()); if (! m_any_packet_yet) { m_last_sample_time = time_now; @@ -191,7 +191,7 @@ void network_throttle::_handle_trafic_exact(size_t packet_size, size_t orginal_s calculate_times_struct cts ; calculate_times(packet_size, cts , false, -1); calculate_times_struct cts2; calculate_times(packet_size, cts2, false, 5); - m_history[0].m_size += packet_size; + m_history.front().m_size += packet_size; std::ostringstream oss; oss << "["; for (auto sample: m_history) oss << sample.m_size << " "; oss << "]" << std::ends; std::string history_str = oss.str(); diff --git a/external/db_drivers/liblmdb/mdb.c b/external/db_drivers/liblmdb/mdb.c index 3b2745f95..8e67d5981 100644 --- a/external/db_drivers/liblmdb/mdb.c +++ b/external/db_drivers/liblmdb/mdb.c @@ -9641,7 +9641,7 @@ mdb_page_split(MDB_cursor *mc, MDB_val *newkey, MDB_val *newdata, pgno_t newpgno * the split so the new page is emptier than the old page. * This yields better packing during sequential inserts. */ - if (nkeys < 20 || nsize > pmax/16 || newindx >= nkeys) { + if (nkeys < 32 || nsize > pmax/16 || newindx >= nkeys) { /* Find split point */ psize = 0; if (newindx <= split_indx || newindx >= nkeys) { diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index a0b62da77..f1454b48e 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -110,6 +110,7 @@ add_subdirectory(checkpoints) add_subdirectory(cryptonote_basic) add_subdirectory(cryptonote_core) add_subdirectory(multisig) +add_subdirectory(net) if(NOT IOS) add_subdirectory(blockchain_db) endif() diff --git a/src/blockchain_db/blockchain_db.cpp b/src/blockchain_db/blockchain_db.cpp index d3eefef6e..041759593 100644 --- a/src/blockchain_db/blockchain_db.cpp +++ b/src/blockchain_db/blockchain_db.cpp @@ -197,6 +197,7 @@ void BlockchainDB::add_transaction(const crypto::hash& blk_hash, const transacti uint64_t BlockchainDB::add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , const std::vector<transaction>& txs @@ -241,7 +242,7 @@ uint64_t BlockchainDB::add_block( const block& blk // call out to subclass implementation to add the block & metadata time1 = epee::misc_utils::get_tick_count(); - add_block(blk, block_weight, cumulative_difficulty, coins_generated, num_rct_outs, blk_hash); + add_block(blk, block_weight, long_term_block_weight, cumulative_difficulty, coins_generated, num_rct_outs, blk_hash); TIME_MEASURE_FINISH(time1); time_add_block1 += time1; diff --git a/src/blockchain_db/blockchain_db.h b/src/blockchain_db/blockchain_db.h index 137d5958a..c3f11ba28 100644 --- a/src/blockchain_db/blockchain_db.h +++ b/src/blockchain_db/blockchain_db.h @@ -358,12 +358,14 @@ private: * * @param blk the block to be added * @param block_weight the weight of the block (transactions and all) + * @param long_term_block_weight the long term block weight of the block (transactions and all) * @param cumulative_difficulty the accumulated difficulty after this block * @param coins_generated the number of coins generated total after this block * @param blk_hash the hash of the block */ virtual void add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , uint64_t num_rct_outs @@ -375,7 +377,7 @@ private: * * The subclass implementing this will remove the block data from the top * block in the chain. The data to be removed is that which was added in - * BlockchainDB::add_block(const block& blk, size_t block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, const crypto::hash& blk_hash) + * BlockchainDB::add_block(const block& blk, size_t block_weight, uint64_t long_term_block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, const crypto::hash& blk_hash) * * If any of this cannot be done, the subclass should throw the corresponding * subclass of DB_EXCEPTION @@ -789,6 +791,7 @@ public: * * @param blk the block to be added * @param block_weight the size of the block (transactions and all) + * @param long_term_block_weight the long term weight of the block (transactions and all) * @param cumulative_difficulty the accumulated difficulty after this block * @param coins_generated the number of coins generated total after this block * @param txs the transactions in the block @@ -797,6 +800,7 @@ public: */ virtual uint64_t add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , const std::vector<transaction>& txs @@ -985,6 +989,17 @@ public: virtual uint64_t get_block_already_generated_coins(const uint64_t& height) const = 0; /** + * @brief fetch a block's long term weight + * + * If the block does not exist, the subclass should throw BLOCK_DNE + * + * @param height the height requested + * + * @return the long term weight + */ + virtual uint64_t get_block_long_term_weight(const uint64_t& height) const = 0; + + /** * @brief fetch a block's hash * * The subclass should return hash of the block with the diff --git a/src/blockchain_db/lmdb/db_lmdb.cpp b/src/blockchain_db/lmdb/db_lmdb.cpp index 8a1303be9..9d2f7821e 100644 --- a/src/blockchain_db/lmdb/db_lmdb.cpp +++ b/src/blockchain_db/lmdb/db_lmdb.cpp @@ -29,6 +29,7 @@ #include <boost/filesystem.hpp> #include <boost/format.hpp> +#include <boost/circular_buffer.hpp> #include <memory> // std::unique_ptr #include <cstring> // memcpy @@ -53,7 +54,7 @@ using epee::string_tools::pod_to_hex; using namespace crypto; // Increase when the DB structure changes -#define VERSION 3 +#define VERSION 4 namespace { @@ -277,7 +278,7 @@ typedef struct mdb_block_info_old crypto::hash bi_hash; } mdb_block_info_old; -typedef struct mdb_block_info +typedef struct mdb_block_info_2 { uint64_t bi_height; uint64_t bi_timestamp; @@ -286,7 +287,21 @@ typedef struct mdb_block_info difficulty_type bi_diff; crypto::hash bi_hash; uint64_t bi_cum_rct; -} mdb_block_info; +} mdb_block_info_2; + +typedef struct mdb_block_info_3 +{ + uint64_t bi_height; + uint64_t bi_timestamp; + uint64_t bi_coins; + uint64_t bi_weight; // a size_t really but we need 32-bit compat + difficulty_type bi_diff; + crypto::hash bi_hash; + uint64_t bi_cum_rct; + uint64_t bi_long_term_block_weight; +} mdb_block_info_3; + +typedef mdb_block_info_3 mdb_block_info; typedef struct blk_height { crypto::hash bh_hash; @@ -504,7 +519,7 @@ void BlockchainLMDB::do_resize(uint64_t increase_size) mdb_env_stat(m_env, &mst); // add 1Gb per resize, instead of doing a percentage increase - uint64_t new_mapsize = (double) mei.me_mapsize + add_size; + uint64_t new_mapsize = (uint64_t) mei.me_mapsize + add_size; // If given, use increase_size instead of above way of resizing. // This is currently used for increasing by an estimated size at start of new @@ -694,7 +709,7 @@ estim: return threshold_size; } -void BlockchainLMDB::add_block(const block& blk, size_t block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, +void BlockchainLMDB::add_block(const block& blk, size_t block_weight, uint64_t long_term_block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, uint64_t num_rct_outs, const crypto::hash& blk_hash) { LOG_PRINT_L3("BlockchainLMDB::" << __func__); @@ -754,6 +769,7 @@ void BlockchainLMDB::add_block(const block& blk, size_t block_weight, const diff const mdb_block_info *bi_prev = (const mdb_block_info*)h.mv_data; bi.bi_cum_rct += bi_prev->bi_cum_rct; } + bi.bi_long_term_block_weight = long_term_block_weight; MDB_val_set(val, bi); result = mdb_cursor_put(m_cur_block_info, (MDB_val *)&zerokval, &val, MDB_APPENDDUP); @@ -1313,14 +1329,14 @@ void BlockchainLMDB::open(const std::string& filename, const int db_flags) MDB_envinfo mei; mdb_env_info(m_env, &mei); - uint64_t cur_mapsize = (double)mei.me_mapsize; + uint64_t cur_mapsize = (uint64_t)mei.me_mapsize; if (cur_mapsize < mapsize) { if (auto result = mdb_env_set_mapsize(m_env, mapsize)) throw0(DB_ERROR(lmdb_error("Failed to set max memory map size: ", result).c_str())); mdb_env_info(m_env, &mei); - cur_mapsize = (double)mei.me_mapsize; + cur_mapsize = (uint64_t)mei.me_mapsize; LOG_PRINT_L1("LMDB memory map size: " << cur_mapsize); } @@ -2486,6 +2502,29 @@ uint64_t BlockchainLMDB::get_block_already_generated_coins(const uint64_t& heigh return ret; } +uint64_t BlockchainLMDB::get_block_long_term_weight(const uint64_t& height) const +{ + LOG_PRINT_L3("BlockchainLMDB::" << __func__); + check_open(); + + TXN_PREFIX_RDONLY(); + RCURSOR(block_info); + + MDB_val_set(result, height); + auto get_result = mdb_cursor_get(m_cur_block_info, (MDB_val *)&zerokval, &result, MDB_GET_BOTH); + if (get_result == MDB_NOTFOUND) + { + throw0(BLOCK_DNE(std::string("Attempt to get block long term weight from height ").append(boost::lexical_cast<std::string>(height)).append(" failed -- block info not in db").c_str())); + } + else if (get_result) + throw0(DB_ERROR("Error attempting to retrieve a long term block weight from the db")); + + mdb_block_info *bi = (mdb_block_info *)result.mv_data; + uint64_t ret = bi->bi_long_term_block_weight; + TXN_POSTFIX_RDONLY(); + return ret; +} + crypto::hash BlockchainLMDB::get_block_hash_from_height(const uint64_t& height) const { LOG_PRINT_L3("BlockchainLMDB::" << __func__); @@ -3524,7 +3563,7 @@ void BlockchainLMDB::block_txn_abort() } } -uint64_t BlockchainLMDB::add_block(const block& blk, size_t block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, +uint64_t BlockchainLMDB::add_block(const block& blk, size_t block_weight, uint64_t long_term_block_weight, const difficulty_type& cumulative_difficulty, const uint64_t& coins_generated, const std::vector<transaction>& txs) { LOG_PRINT_L3("BlockchainLMDB::" << __func__); @@ -3543,7 +3582,7 @@ uint64_t BlockchainLMDB::add_block(const block& blk, size_t block_weight, const try { - BlockchainDB::add_block(blk, block_weight, cumulative_difficulty, coins_generated, txs); + BlockchainDB::add_block(blk, block_weight, long_term_block_weight, cumulative_difficulty, coins_generated, txs); } catch (const DB_ERROR_TXN_START& e) { @@ -4748,6 +4787,7 @@ void BlockchainLMDB::migrate_2_3() throw0(DB_ERROR(lmdb_error("Failed to delete old block_info table: ", result).c_str())); RENAME_DB("block_infn"); + mdb_dbi_close(m_env, m_block_info); lmdb_db_open(txn, "block_info", MDB_INTEGERKEY | MDB_CREATE | MDB_DUPSORT | MDB_DUPFIXED, m_block_info, "Failed to open db handle for block_infn"); mdb_set_dupsort(txn, m_block_info, compare_uint64); @@ -4768,6 +4808,166 @@ void BlockchainLMDB::migrate_2_3() txn.commit(); } +void BlockchainLMDB::migrate_3_4() +{ + LOG_PRINT_L3("BlockchainLMDB::" << __func__); + uint64_t i; + int result; + mdb_txn_safe txn(false); + MDB_val k, v; + char *ptr; + bool past_long_term_weight = false; + + MGINFO_YELLOW("Migrating blockchain from DB version 3 to 4 - this may take a while:"); + + do { + LOG_PRINT_L1("migrating block info:"); + + result = mdb_txn_begin(m_env, NULL, 0, txn); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str())); + + MDB_stat db_stats; + if ((result = mdb_stat(txn, m_blocks, &db_stats))) + throw0(DB_ERROR(lmdb_error("Failed to query m_blocks: ", result).c_str())); + const uint64_t blockchain_height = db_stats.ms_entries; + + boost::circular_buffer<uint64_t> long_term_block_weights(CRYPTONOTE_LONG_TERM_BLOCK_WEIGHT_WINDOW_SIZE); + + /* the block_info table name is the same but the old version and new version + * have incompatible data. Create a new table. We want the name to be similar + * to the old name so that it will occupy the same location in the DB. + */ + MDB_dbi o_block_info = m_block_info; + lmdb_db_open(txn, "block_infn", MDB_INTEGERKEY | MDB_CREATE | MDB_DUPSORT | MDB_DUPFIXED, m_block_info, "Failed to open db handle for block_infn"); + mdb_set_dupsort(txn, m_block_info, compare_uint64); + + + MDB_cursor *c_blocks; + result = mdb_cursor_open(txn, m_blocks, &c_blocks); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to open a cursor for blocks: ", result).c_str())); + + MDB_cursor *c_old, *c_cur; + i = 0; + while(1) { + if (!(i % 1000)) { + if (i) { + LOGIF(el::Level::Info) { + std::cout << i << " / " << blockchain_height << " \r" << std::flush; + } + txn.commit(); + result = mdb_txn_begin(m_env, NULL, 0, txn); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str())); + } + result = mdb_cursor_open(txn, m_block_info, &c_cur); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to open a cursor for block_infn: ", result).c_str())); + result = mdb_cursor_open(txn, o_block_info, &c_old); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to open a cursor for block_info: ", result).c_str())); + result = mdb_cursor_open(txn, m_blocks, &c_blocks); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to open a cursor for blocks: ", result).c_str())); + if (!i) { + MDB_stat db_stat; + result = mdb_stat(txn, m_block_info, &db_stats); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to query m_block_info: ", result).c_str())); + i = db_stats.ms_entries; + } + } + result = mdb_cursor_get(c_old, &k, &v, MDB_NEXT); + if (result == MDB_NOTFOUND) { + txn.commit(); + break; + } + else if (result) + throw0(DB_ERROR(lmdb_error("Failed to get a record from block_info: ", result).c_str())); + const mdb_block_info_2 *bi_old = (const mdb_block_info_2*)v.mv_data; + mdb_block_info_3 bi; + bi.bi_height = bi_old->bi_height; + bi.bi_timestamp = bi_old->bi_timestamp; + bi.bi_coins = bi_old->bi_coins; + bi.bi_weight = bi_old->bi_weight; + bi.bi_diff = bi_old->bi_diff; + bi.bi_hash = bi_old->bi_hash; + bi.bi_cum_rct = bi_old->bi_cum_rct; + + // get block major version to determine which rule is in place + if (!past_long_term_weight) + { + MDB_val_copy<uint64_t> kb(bi.bi_height); + MDB_val vb; + result = mdb_cursor_get(c_blocks, &kb, &vb, MDB_SET); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to query m_blocks: ", result).c_str())); + if (vb.mv_size == 0) + throw0(DB_ERROR("Invalid data from m_blocks")); + const uint8_t block_major_version = *((const uint8_t*)vb.mv_data); + if (block_major_version >= HF_VERSION_LONG_TERM_BLOCK_WEIGHT) + past_long_term_weight = true; + } + + uint64_t long_term_block_weight; + if (past_long_term_weight) + { + std::vector<uint64_t> weights(long_term_block_weights.begin(), long_term_block_weights.end()); + uint64_t long_term_effective_block_median_weight = std::max<uint64_t>(CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5, epee::misc_utils::median(weights)); + long_term_block_weight = std::min<uint64_t>(bi.bi_weight, long_term_effective_block_median_weight + long_term_effective_block_median_weight * 2 / 5); + } + else + { + long_term_block_weight = bi.bi_weight; + } + long_term_block_weights.push_back(long_term_block_weight); + bi.bi_long_term_block_weight = long_term_block_weight; + + MDB_val_set(nv, bi); + result = mdb_cursor_put(c_cur, (MDB_val *)&zerokval, &nv, MDB_APPENDDUP); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to put a record into block_infn: ", result).c_str())); + /* we delete the old records immediately, so the overall DB and mapsize should not grow. + * This is a little slower than just letting mdb_drop() delete it all at the end, but + * it saves a significant amount of disk space. + */ + result = mdb_cursor_del(c_old, 0); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to delete a record from block_info: ", result).c_str())); + i++; + } + + result = mdb_txn_begin(m_env, NULL, 0, txn); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str())); + /* Delete the old table */ + result = mdb_drop(txn, o_block_info, 1); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to delete old block_info table: ", result).c_str())); + + RENAME_DB("block_infn"); + mdb_dbi_close(m_env, m_block_info); + + lmdb_db_open(txn, "block_info", MDB_INTEGERKEY | MDB_CREATE | MDB_DUPSORT | MDB_DUPFIXED, m_block_info, "Failed to open db handle for block_infn"); + mdb_set_dupsort(txn, m_block_info, compare_uint64); + + txn.commit(); + } while(0); + + uint32_t version = 4; + v.mv_data = (void *)&version; + v.mv_size = sizeof(version); + MDB_val_str(vk, "version"); + result = mdb_txn_begin(m_env, NULL, 0, txn); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to create a transaction for the db: ", result).c_str())); + result = mdb_put(txn, m_properties, &vk, &v, 0); + if (result) + throw0(DB_ERROR(lmdb_error("Failed to update version for the db: ", result).c_str())); + txn.commit(); +} + void BlockchainLMDB::migrate(const uint32_t oldversion) { if (oldversion < 1) @@ -4776,6 +4976,8 @@ void BlockchainLMDB::migrate(const uint32_t oldversion) migrate_1_2(); if (oldversion < 3) migrate_2_3(); + if (oldversion < 4) + migrate_3_4(); } } // namespace cryptonote diff --git a/src/blockchain_db/lmdb/db_lmdb.h b/src/blockchain_db/lmdb/db_lmdb.h index c07ab8da5..5764f9ae4 100644 --- a/src/blockchain_db/lmdb/db_lmdb.h +++ b/src/blockchain_db/lmdb/db_lmdb.h @@ -225,6 +225,8 @@ public: virtual uint64_t get_block_already_generated_coins(const uint64_t& height) const; + virtual uint64_t get_block_long_term_weight(const uint64_t& height) const; + virtual crypto::hash get_block_hash_from_height(const uint64_t& height) const; virtual std::vector<block> get_blocks_range(const uint64_t& h1, const uint64_t& h2) const; @@ -292,6 +294,7 @@ public: virtual uint64_t add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , const std::vector<transaction>& txs @@ -341,6 +344,7 @@ private: virtual void add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , uint64_t num_rct_outs @@ -405,6 +409,9 @@ private: // migrate from DB version 2 to 3 void migrate_2_3(); + // migrate from DB version 3 to 4 + void migrate_3_4(); + void cleanup_batch(); private: diff --git a/tests/unit_tests/testdb.h b/src/blockchain_db/testdb.h index 8f5cf70e8..35dfbe673 100644 --- a/tests/unit_tests/testdb.h +++ b/src/blockchain_db/testdb.h @@ -33,9 +33,11 @@ #include <string> #include <vector> #include <map> -#include "gtest/gtest.h" -#include "blockchain_db/blockchain_db.h" +#include "blockchain_db.h" + +namespace cryptonote +{ class BaseTestDB: public cryptonote::BlockchainDB { public: @@ -73,6 +75,7 @@ public: virtual cryptonote::difficulty_type get_block_cumulative_difficulty(const uint64_t& height) const { return 10; } virtual cryptonote::difficulty_type get_block_difficulty(const uint64_t& height) const { return 0; } virtual uint64_t get_block_already_generated_coins(const uint64_t& height) const { return 10000000000; } + virtual uint64_t get_block_long_term_weight(const uint64_t& height) const { return 128; } virtual crypto::hash get_block_hash_from_height(const uint64_t& height) const { return crypto::hash(); } virtual std::vector<cryptonote::block> get_blocks_range(const uint64_t& h1, const uint64_t& h2) const { return std::vector<cryptonote::block>(); } virtual std::vector<crypto::hash> get_hashes_range(const uint64_t& h1, const uint64_t& h2) const { return std::vector<crypto::hash>(); } @@ -128,6 +131,7 @@ public: virtual void add_block( const cryptonote::block& blk , size_t block_weight + , uint64_t long_term_block_weight , const cryptonote::difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , uint64_t num_rct_outs @@ -145,3 +149,4 @@ public: virtual void prune_outputs(uint64_t amount) {} }; +} diff --git a/src/blockchain_utilities/blockchain_import.cpp b/src/blockchain_utilities/blockchain_import.cpp index 2a8a6f494..e6ec20c3b 100644 --- a/src/blockchain_utilities/blockchain_import.cpp +++ b/src/blockchain_utilities/blockchain_import.cpp @@ -485,7 +485,8 @@ int import_from_file(cryptonote::core& core, const std::string& import_file_path try { - core.get_blockchain_storage().get_db().add_block(b, block_weight, cumulative_difficulty, coins_generated, txs); + uint64_t long_term_block_weight = core.get_blockchain_storage().get_next_long_term_block_weight(block_weight); + core.get_blockchain_storage().get_db().add_block(b, block_weight, long_term_block_weight, cumulative_difficulty, coins_generated, txs); } catch (const std::exception& e) { diff --git a/src/blockchain_utilities/blockchain_prune.cpp b/src/blockchain_utilities/blockchain_prune.cpp index 8e13f2c04..36080aade 100644 --- a/src/blockchain_utilities/blockchain_prune.cpp +++ b/src/blockchain_utilities/blockchain_prune.cpp @@ -611,6 +611,24 @@ int main(int argc, char* argv[]) } already_pruned = true; } + if (n == 0) + { + const uint64_t blockchain_height = core_storage[0]->get_current_blockchain_height(); + const crypto::hash hash = core_storage[0]->get_block_id_by_height(blockchain_height - 1); + cryptonote::block block; + if (core_storage[0]->get_block_by_hash(hash, block)) + { + if (block.major_version < 10) + { + time_t now = time(NULL); + if (now < 1555286400) // 15 april 2019 + { + MERROR("Pruning before v10 will confuse peers. Wait for v10 first"); + return 1; + } + } + } + } } core_storage[0]->deinit(); core_storage[0].reset(NULL); diff --git a/src/common/CMakeLists.txt b/src/common/CMakeLists.txt index 212a1891e..bcf9cbce7 100644 --- a/src/common/CMakeLists.txt +++ b/src/common/CMakeLists.txt @@ -45,6 +45,7 @@ set(common_sources threadpool.cpp updates.cpp aligned.c + timings.cc combinator.cpp) if (STACK_TRACE) @@ -84,6 +85,7 @@ set(common_private_headers threadpool.h updates.h aligned.h + timings.h combinator.h) monero_private_headers(common diff --git a/src/common/download.cpp b/src/common/download.cpp index 58ce0595f..7c38cfa5b 100644 --- a/src/common/download.cpp +++ b/src/common/download.cpp @@ -179,8 +179,8 @@ namespace tools lock.unlock(); - bool ssl = u_c.schema == "https"; - uint16_t port = u_c.port ? u_c.port : ssl ? 443 : 80; + epee::net_utils::ssl_support_t ssl = u_c.schema == "https" ? epee::net_utils::ssl_support_t::e_ssl_support_enabled : epee::net_utils::ssl_support_t::e_ssl_support_disabled; + uint16_t port = u_c.port ? u_c.port : ssl == epee::net_utils::ssl_support_t::e_ssl_support_enabled ? 443 : 80; MDEBUG("Connecting to " << u_c.host << ":" << port); client.set_server(u_c.host, std::to_string(port), boost::none, ssl); if (!client.connect(std::chrono::seconds(30))) diff --git a/src/common/notify.cpp b/src/common/notify.cpp index c3165fb05..e2df5096d 100644 --- a/src/common/notify.cpp +++ b/src/common/notify.cpp @@ -48,7 +48,7 @@ Notify::Notify(const char *spec) { CHECK_AND_ASSERT_THROW_MES(spec, "Null spec"); - boost::split(args, spec, boost::is_any_of(" ")); + boost::split(args, spec, boost::is_any_of(" \t"), boost::token_compress_on); CHECK_AND_ASSERT_THROW_MES(args.size() > 0, "Failed to parse spec"); if (strchr(spec, '\'') || strchr(spec, '\"') || strchr(spec, '\\')) MWARNING("A notification spec contains a quote or backslash: note that these are handled verbatim, which may not be the intent"); diff --git a/src/common/perf_timer.h b/src/common/perf_timer.h index d859cf576..5203da205 100644 --- a/src/common/perf_timer.h +++ b/src/common/perf_timer.h @@ -53,6 +53,7 @@ public: void resume(); void reset(); uint64_t value() const; + operator uint64_t() const { return value(); } protected: uint64_t ticks; diff --git a/src/common/timings.cc b/src/common/timings.cc new file mode 100644 index 000000000..cb8deff2a --- /dev/null +++ b/src/common/timings.cc @@ -0,0 +1,125 @@ +#include <string.h> +#include <error.h> +#include <time.h> +#include <algorithm> +#include <boost/algorithm/string.hpp> +#include "misc_log_ex.h" +#include "timings.h" + +#define N_EXPECTED_FIELDS (8+11) + +TimingsDatabase::TimingsDatabase() +{ +} + +TimingsDatabase::TimingsDatabase(const std::string &filename): + filename(filename) +{ + load(); +} + +TimingsDatabase::~TimingsDatabase() +{ + save(); +} + +bool TimingsDatabase::load() +{ + instances.clear(); + + if (filename.empty()) + return true; + + FILE *f = fopen(filename.c_str(), "r"); + if (!f) + { + MDEBUG("Failed to load timings file " << filename << ": " << strerror(errno)); + return false; + } + while (1) + { + char s[4096]; + if (!fgets(s, sizeof(s), f)) + break; + char *tab = strchr(s, '\t'); + if (!tab) + { + MWARNING("Bad format: no tab found"); + continue; + } + const std::string name = std::string(s, tab - s); + std::vector<std::string> fields; + char *ptr = tab + 1; + boost::split(fields, ptr, boost::is_any_of(" ")); + if (fields.size() != N_EXPECTED_FIELDS) + { + MERROR("Bad format: wrong number of fields: got " << fields.size() << " expected " << N_EXPECTED_FIELDS); + continue; + } + + instance i; + + unsigned int idx = 0; + i.t = atoi(fields[idx++].c_str()); + i.npoints = atoi(fields[idx++].c_str()); + i.min = atof(fields[idx++].c_str()); + i.max = atof(fields[idx++].c_str()); + i.mean = atof(fields[idx++].c_str()); + i.median = atof(fields[idx++].c_str()); + i.stddev = atof(fields[idx++].c_str()); + i.npskew = atof(fields[idx++].c_str()); + i.deciles.reserve(11); + for (int n = 0; n < 11; ++n) + { + i.deciles.push_back(atoi(fields[idx++].c_str())); + } + instances.insert(std::make_pair(name, i)); + } + fclose(f); + return true; +} + +bool TimingsDatabase::save() +{ + if (filename.empty()) + return true; + + FILE *f = fopen(filename.c_str(), "w"); + if (!f) + { + MERROR("Failed to write to file " << filename << ": " << strerror(errno)); + return false; + } + for (const auto &i: instances) + { + fprintf(f, "%s", i.first.c_str()); + fprintf(f, "\t%lu", (unsigned long)i.second.t); + fprintf(f, " %zu", i.second.npoints); + fprintf(f, " %f", i.second.min); + fprintf(f, " %f", i.second.max); + fprintf(f, " %f", i.second.mean); + fprintf(f, " %f", i.second.median); + fprintf(f, " %f", i.second.stddev); + fprintf(f, " %f", i.second.npskew); + for (uint64_t v: i.second.deciles) + fprintf(f, " %lu", (unsigned long)v); + fputc('\n', f); + } + fclose(f); + return true; +} + +std::vector<TimingsDatabase::instance> TimingsDatabase::get(const char *name) const +{ + std::vector<instance> ret; + auto range = instances.equal_range(name); + for (auto i = range.first; i != range.second; ++i) + ret.push_back(i->second); + std::sort(ret.begin(), ret.end(), [](const instance &e0, const instance &e1){ return e0.t < e1.t; }); + return ret; +} + +void TimingsDatabase::add(const char *name, const instance &i) +{ + instances.insert(std::make_pair(name, i)); +} diff --git a/src/common/timings.h b/src/common/timings.h new file mode 100644 index 000000000..fb905611f --- /dev/null +++ b/src/common/timings.h @@ -0,0 +1,34 @@ +#pragma once + +#include <stdint.h> +#include <string> +#include <vector> +#include <map> + +class TimingsDatabase +{ +public: + struct instance + { + time_t t; + size_t npoints; + double min, max, mean, median, stddev, npskew; + std::vector<uint64_t> deciles; + }; + +public: + TimingsDatabase(); + TimingsDatabase(const std::string &filename); + ~TimingsDatabase(); + + std::vector<instance> get(const char *name) const; + void add(const char *name, const instance &data); + +private: + bool load(); + bool save(); + +private: + std::string filename; + std::multimap<std::string, instance> instances; +}; diff --git a/src/crypto/CMakeLists.txt b/src/crypto/CMakeLists.txt index 0c635e7cb..5ce43be22 100644 --- a/src/crypto/CMakeLists.txt +++ b/src/crypto/CMakeLists.txt @@ -45,6 +45,8 @@ set(crypto_sources random.c skein.c slow-hash.c + CryptonightR_JIT.c + CryptonightR_template.S tree-hash.c) set(crypto_headers) @@ -66,7 +68,9 @@ set(crypto_private_headers oaes_lib.h random.h skein.h - skein_port.h) + skein_port.h + CryptonightR_JIT.h + CryptonightR_template.h) monero_private_headers(cncrypto ${crypto_private_headers}) @@ -101,4 +105,5 @@ if (ANDROID OR IOS) endif() endif() - +# cheat because cmake and ccache hate each other +set_property(SOURCE CryptonightR_template.S PROPERTY LANGUAGE C) diff --git a/src/crypto/CryptonightR_JIT.c b/src/crypto/CryptonightR_JIT.c new file mode 100644 index 000000000..9add65296 --- /dev/null +++ b/src/crypto/CryptonightR_JIT.c @@ -0,0 +1,102 @@ +#include <assert.h> +#include <stddef.h> +#include <stdint.h> +#include <string.h> +#include <stdio.h> +#include <unistd.h> + +#include "int-util.h" +#include "hash-ops.h" +#include "variant4_random_math.h" +#include "CryptonightR_JIT.h" +#include "CryptonightR_template.h" + +static const uint8_t prologue[] = { + 0x4C, 0x8B, 0xD7, // mov r10, rdi + 0x53, // push rbx + 0x55, // push rbp + 0x41, 0x57, // push r15 + 0x4C, 0x8B, 0xDC, // mov r11, rsp + 0x41, 0x8B, 0x1A, // mov ebx, DWORD PTR [r10] + 0x41, 0x8B, 0x72, 0x04, // mov esi, DWORD PTR [r10+4] + 0x41, 0x8B, 0x7A, 0x08, // mov edi, DWORD PTR [r10+8] + 0x41, 0x8B, 0x6A, 0x0C, // mov ebp, DWORD PTR [r10+12] + 0x41, 0x8B, 0x62, 0x10, // mov esp, DWORD PTR [r10+16] + 0x45, 0x8B, 0x7A, 0x14, // mov r15d, DWORD PTR [r10+20] + 0x41, 0x8B, 0x42, 0x18, // mov eax, DWORD PTR [r10+24] + 0x41, 0x8B, 0x52, 0x1C, // mov edx, DWORD PTR [r10+28] + 0x45, 0x8B, 0x4A, 0x20, // mov r9d, DWORD PTR [r10+32] +}; + +static const uint8_t epilogue[] = { + 0x49, 0x8B, 0xE3, // mov rsp, r11 + 0x41, 0x89, 0x1A, // mov DWORD PTR [r10], ebx + 0x41, 0x89, 0x72, 0x04, // mov DWORD PTR [r10+4], esi + 0x41, 0x89, 0x7A, 0x08, // mov DWORD PTR [r10+8], edi + 0x41, 0x89, 0x6A, 0x0C, // mov DWORD PTR [r10+12], ebp + 0x41, 0x5F, // pop r15 + 0x5D, // pop rbp + 0x5B, // pop rbx + 0xC3, // ret +}; + +#define APPEND_CODE(src, size) \ + do { \ + if (JIT_code + (size) > JIT_code_end) \ + return -1; \ + memcpy(JIT_code, (src), (size)); \ + JIT_code += (size); \ + } while (0) + +int v4_generate_JIT_code(const struct V4_Instruction* code, v4_random_math_JIT_func buf, const size_t buf_size) +{ + uint8_t* JIT_code = (uint8_t*) buf; + const uint8_t* JIT_code_end = JIT_code + buf_size; + + APPEND_CODE(prologue, sizeof(prologue)); + + uint32_t prev_rot_src = 0xFFFFFFFFU; + + for (int i = 0;; ++i) + { + const struct V4_Instruction inst = code[i]; + if (inst.opcode == RET) + break; + + const uint8_t opcode = (inst.opcode == MUL) ? inst.opcode : (inst.opcode + 2); + + const uint32_t a = inst.dst_index; + const uint32_t b = inst.src_index; + const uint8_t c = opcode | (inst.dst_index << V4_OPCODE_BITS) | (((inst.src_index == 8) ? inst.dst_index : inst.src_index) << (V4_OPCODE_BITS + V4_DST_INDEX_BITS)); + + switch (inst.opcode) + { + case ROR: + case ROL: + if (b != prev_rot_src) + { + prev_rot_src = b; + const uint8_t* p1 = (const uint8_t*) instructions_mov[c]; + const uint8_t* p2 = (const uint8_t*) instructions_mov[c + 1]; + APPEND_CODE(p1, p2 - p1); + } + break; + } + + if (a == prev_rot_src) + prev_rot_src = 0xFFFFFFFFU; + + const uint8_t* p1 = (const uint8_t*) instructions[c]; + const uint8_t* p2 = (const uint8_t*) instructions[c + 1]; + APPEND_CODE(p1, p2 - p1); + + if (inst.opcode == ADD) + *(uint32_t*)(JIT_code - 4) = inst.C; + } + + APPEND_CODE(epilogue, sizeof(epilogue)); + + __builtin___clear_cache((char*)buf, (char*)JIT_code); + + return 0; +} diff --git a/src/crypto/CryptonightR_JIT.h b/src/crypto/CryptonightR_JIT.h new file mode 100644 index 000000000..5f689b37b --- /dev/null +++ b/src/crypto/CryptonightR_JIT.h @@ -0,0 +1,18 @@ +#ifndef CRYPTONIGHTR_JIT_H +#define CRYPTONIGHTR_JIT_H + +// Minimalistic JIT code generator for random math sequence in CryptonightR +// +// Usage: +// - Allocate writable and executable memory +// - Call v4_generate_JIT_code with "buf" pointed to memory allocated on previous step +// - Call the generated code instead of "v4_random_math(code, r)", omit the "code" parameter + +typedef void (*v4_random_math_JIT_func)(uint32_t* r) __attribute__((sysv_abi)); + +// Given the random math sequence, generates machine code (x86-64) for it +// Returns 0 if code was generated successfully +// Returns -1 if provided buffer was too small +int v4_generate_JIT_code(const struct V4_Instruction* code, v4_random_math_JIT_func buf, const size_t buf_size); + +#endif // CRYPTONIGHTR_JIT_H diff --git a/src/crypto/CryptonightR_template.S b/src/crypto/CryptonightR_template.S new file mode 100644 index 000000000..068de22ec --- /dev/null +++ b/src/crypto/CryptonightR_template.S @@ -0,0 +1,1590 @@ +#ifdef __APPLE__ +# define ALIGN(x) .align 6 +#else +# define ALIGN(x) .align 64 +#endif +.intel_syntax noprefix +#ifdef __APPLE__ +# define FN_PREFIX(fn) _ ## fn +.text +#else +# define FN_PREFIX(fn) fn +.section .text +#endif + +#define PUBLIC .global + +PUBLIC FN_PREFIX(CryptonightR_instruction0) +PUBLIC FN_PREFIX(CryptonightR_instruction1) +PUBLIC FN_PREFIX(CryptonightR_instruction2) +PUBLIC FN_PREFIX(CryptonightR_instruction3) +PUBLIC FN_PREFIX(CryptonightR_instruction4) +PUBLIC FN_PREFIX(CryptonightR_instruction5) +PUBLIC FN_PREFIX(CryptonightR_instruction6) +PUBLIC FN_PREFIX(CryptonightR_instruction7) +PUBLIC FN_PREFIX(CryptonightR_instruction8) +PUBLIC FN_PREFIX(CryptonightR_instruction9) +PUBLIC FN_PREFIX(CryptonightR_instruction10) +PUBLIC FN_PREFIX(CryptonightR_instruction11) +PUBLIC FN_PREFIX(CryptonightR_instruction12) +PUBLIC FN_PREFIX(CryptonightR_instruction13) +PUBLIC FN_PREFIX(CryptonightR_instruction14) +PUBLIC FN_PREFIX(CryptonightR_instruction15) +PUBLIC FN_PREFIX(CryptonightR_instruction16) +PUBLIC FN_PREFIX(CryptonightR_instruction17) +PUBLIC FN_PREFIX(CryptonightR_instruction18) +PUBLIC FN_PREFIX(CryptonightR_instruction19) +PUBLIC FN_PREFIX(CryptonightR_instruction20) +PUBLIC FN_PREFIX(CryptonightR_instruction21) +PUBLIC FN_PREFIX(CryptonightR_instruction22) +PUBLIC FN_PREFIX(CryptonightR_instruction23) +PUBLIC FN_PREFIX(CryptonightR_instruction24) +PUBLIC FN_PREFIX(CryptonightR_instruction25) +PUBLIC FN_PREFIX(CryptonightR_instruction26) +PUBLIC FN_PREFIX(CryptonightR_instruction27) +PUBLIC FN_PREFIX(CryptonightR_instruction28) +PUBLIC FN_PREFIX(CryptonightR_instruction29) +PUBLIC FN_PREFIX(CryptonightR_instruction30) +PUBLIC FN_PREFIX(CryptonightR_instruction31) +PUBLIC FN_PREFIX(CryptonightR_instruction32) +PUBLIC FN_PREFIX(CryptonightR_instruction33) +PUBLIC FN_PREFIX(CryptonightR_instruction34) +PUBLIC FN_PREFIX(CryptonightR_instruction35) +PUBLIC FN_PREFIX(CryptonightR_instruction36) +PUBLIC FN_PREFIX(CryptonightR_instruction37) +PUBLIC FN_PREFIX(CryptonightR_instruction38) +PUBLIC FN_PREFIX(CryptonightR_instruction39) +PUBLIC FN_PREFIX(CryptonightR_instruction40) +PUBLIC FN_PREFIX(CryptonightR_instruction41) +PUBLIC FN_PREFIX(CryptonightR_instruction42) +PUBLIC FN_PREFIX(CryptonightR_instruction43) +PUBLIC FN_PREFIX(CryptonightR_instruction44) +PUBLIC FN_PREFIX(CryptonightR_instruction45) +PUBLIC FN_PREFIX(CryptonightR_instruction46) +PUBLIC FN_PREFIX(CryptonightR_instruction47) +PUBLIC FN_PREFIX(CryptonightR_instruction48) +PUBLIC FN_PREFIX(CryptonightR_instruction49) +PUBLIC FN_PREFIX(CryptonightR_instruction50) +PUBLIC FN_PREFIX(CryptonightR_instruction51) +PUBLIC FN_PREFIX(CryptonightR_instruction52) +PUBLIC FN_PREFIX(CryptonightR_instruction53) +PUBLIC FN_PREFIX(CryptonightR_instruction54) +PUBLIC FN_PREFIX(CryptonightR_instruction55) +PUBLIC FN_PREFIX(CryptonightR_instruction56) +PUBLIC FN_PREFIX(CryptonightR_instruction57) +PUBLIC FN_PREFIX(CryptonightR_instruction58) +PUBLIC FN_PREFIX(CryptonightR_instruction59) +PUBLIC FN_PREFIX(CryptonightR_instruction60) +PUBLIC FN_PREFIX(CryptonightR_instruction61) +PUBLIC FN_PREFIX(CryptonightR_instruction62) +PUBLIC FN_PREFIX(CryptonightR_instruction63) +PUBLIC FN_PREFIX(CryptonightR_instruction64) +PUBLIC FN_PREFIX(CryptonightR_instruction65) +PUBLIC FN_PREFIX(CryptonightR_instruction66) +PUBLIC FN_PREFIX(CryptonightR_instruction67) +PUBLIC FN_PREFIX(CryptonightR_instruction68) +PUBLIC FN_PREFIX(CryptonightR_instruction69) +PUBLIC FN_PREFIX(CryptonightR_instruction70) +PUBLIC FN_PREFIX(CryptonightR_instruction71) +PUBLIC FN_PREFIX(CryptonightR_instruction72) +PUBLIC FN_PREFIX(CryptonightR_instruction73) +PUBLIC FN_PREFIX(CryptonightR_instruction74) +PUBLIC FN_PREFIX(CryptonightR_instruction75) +PUBLIC FN_PREFIX(CryptonightR_instruction76) +PUBLIC FN_PREFIX(CryptonightR_instruction77) +PUBLIC FN_PREFIX(CryptonightR_instruction78) +PUBLIC FN_PREFIX(CryptonightR_instruction79) +PUBLIC FN_PREFIX(CryptonightR_instruction80) +PUBLIC FN_PREFIX(CryptonightR_instruction81) +PUBLIC FN_PREFIX(CryptonightR_instruction82) +PUBLIC FN_PREFIX(CryptonightR_instruction83) +PUBLIC FN_PREFIX(CryptonightR_instruction84) +PUBLIC FN_PREFIX(CryptonightR_instruction85) +PUBLIC FN_PREFIX(CryptonightR_instruction86) +PUBLIC FN_PREFIX(CryptonightR_instruction87) +PUBLIC FN_PREFIX(CryptonightR_instruction88) +PUBLIC FN_PREFIX(CryptonightR_instruction89) +PUBLIC FN_PREFIX(CryptonightR_instruction90) +PUBLIC FN_PREFIX(CryptonightR_instruction91) +PUBLIC FN_PREFIX(CryptonightR_instruction92) +PUBLIC FN_PREFIX(CryptonightR_instruction93) +PUBLIC FN_PREFIX(CryptonightR_instruction94) +PUBLIC FN_PREFIX(CryptonightR_instruction95) +PUBLIC FN_PREFIX(CryptonightR_instruction96) +PUBLIC FN_PREFIX(CryptonightR_instruction97) +PUBLIC FN_PREFIX(CryptonightR_instruction98) +PUBLIC FN_PREFIX(CryptonightR_instruction99) +PUBLIC FN_PREFIX(CryptonightR_instruction100) +PUBLIC FN_PREFIX(CryptonightR_instruction101) +PUBLIC FN_PREFIX(CryptonightR_instruction102) +PUBLIC FN_PREFIX(CryptonightR_instruction103) +PUBLIC FN_PREFIX(CryptonightR_instruction104) +PUBLIC FN_PREFIX(CryptonightR_instruction105) +PUBLIC FN_PREFIX(CryptonightR_instruction106) +PUBLIC FN_PREFIX(CryptonightR_instruction107) +PUBLIC FN_PREFIX(CryptonightR_instruction108) +PUBLIC FN_PREFIX(CryptonightR_instruction109) +PUBLIC FN_PREFIX(CryptonightR_instruction110) +PUBLIC FN_PREFIX(CryptonightR_instruction111) +PUBLIC FN_PREFIX(CryptonightR_instruction112) +PUBLIC FN_PREFIX(CryptonightR_instruction113) +PUBLIC FN_PREFIX(CryptonightR_instruction114) +PUBLIC FN_PREFIX(CryptonightR_instruction115) +PUBLIC FN_PREFIX(CryptonightR_instruction116) +PUBLIC FN_PREFIX(CryptonightR_instruction117) +PUBLIC FN_PREFIX(CryptonightR_instruction118) +PUBLIC FN_PREFIX(CryptonightR_instruction119) +PUBLIC FN_PREFIX(CryptonightR_instruction120) +PUBLIC FN_PREFIX(CryptonightR_instruction121) +PUBLIC FN_PREFIX(CryptonightR_instruction122) +PUBLIC FN_PREFIX(CryptonightR_instruction123) +PUBLIC FN_PREFIX(CryptonightR_instruction124) +PUBLIC FN_PREFIX(CryptonightR_instruction125) +PUBLIC FN_PREFIX(CryptonightR_instruction126) +PUBLIC FN_PREFIX(CryptonightR_instruction127) +PUBLIC FN_PREFIX(CryptonightR_instruction128) +PUBLIC FN_PREFIX(CryptonightR_instruction129) +PUBLIC FN_PREFIX(CryptonightR_instruction130) +PUBLIC FN_PREFIX(CryptonightR_instruction131) +PUBLIC FN_PREFIX(CryptonightR_instruction132) +PUBLIC FN_PREFIX(CryptonightR_instruction133) +PUBLIC FN_PREFIX(CryptonightR_instruction134) +PUBLIC FN_PREFIX(CryptonightR_instruction135) +PUBLIC FN_PREFIX(CryptonightR_instruction136) +PUBLIC FN_PREFIX(CryptonightR_instruction137) +PUBLIC FN_PREFIX(CryptonightR_instruction138) +PUBLIC FN_PREFIX(CryptonightR_instruction139) +PUBLIC FN_PREFIX(CryptonightR_instruction140) +PUBLIC FN_PREFIX(CryptonightR_instruction141) +PUBLIC FN_PREFIX(CryptonightR_instruction142) +PUBLIC FN_PREFIX(CryptonightR_instruction143) +PUBLIC FN_PREFIX(CryptonightR_instruction144) +PUBLIC FN_PREFIX(CryptonightR_instruction145) +PUBLIC FN_PREFIX(CryptonightR_instruction146) +PUBLIC FN_PREFIX(CryptonightR_instruction147) +PUBLIC FN_PREFIX(CryptonightR_instruction148) +PUBLIC FN_PREFIX(CryptonightR_instruction149) +PUBLIC FN_PREFIX(CryptonightR_instruction150) +PUBLIC FN_PREFIX(CryptonightR_instruction151) +PUBLIC FN_PREFIX(CryptonightR_instruction152) +PUBLIC FN_PREFIX(CryptonightR_instruction153) +PUBLIC FN_PREFIX(CryptonightR_instruction154) +PUBLIC FN_PREFIX(CryptonightR_instruction155) +PUBLIC FN_PREFIX(CryptonightR_instruction156) +PUBLIC FN_PREFIX(CryptonightR_instruction157) +PUBLIC FN_PREFIX(CryptonightR_instruction158) +PUBLIC FN_PREFIX(CryptonightR_instruction159) +PUBLIC FN_PREFIX(CryptonightR_instruction160) +PUBLIC FN_PREFIX(CryptonightR_instruction161) +PUBLIC FN_PREFIX(CryptonightR_instruction162) +PUBLIC FN_PREFIX(CryptonightR_instruction163) +PUBLIC FN_PREFIX(CryptonightR_instruction164) +PUBLIC FN_PREFIX(CryptonightR_instruction165) +PUBLIC FN_PREFIX(CryptonightR_instruction166) +PUBLIC FN_PREFIX(CryptonightR_instruction167) +PUBLIC FN_PREFIX(CryptonightR_instruction168) +PUBLIC FN_PREFIX(CryptonightR_instruction169) +PUBLIC FN_PREFIX(CryptonightR_instruction170) +PUBLIC FN_PREFIX(CryptonightR_instruction171) +PUBLIC FN_PREFIX(CryptonightR_instruction172) +PUBLIC FN_PREFIX(CryptonightR_instruction173) +PUBLIC FN_PREFIX(CryptonightR_instruction174) +PUBLIC FN_PREFIX(CryptonightR_instruction175) +PUBLIC FN_PREFIX(CryptonightR_instruction176) +PUBLIC FN_PREFIX(CryptonightR_instruction177) +PUBLIC FN_PREFIX(CryptonightR_instruction178) +PUBLIC FN_PREFIX(CryptonightR_instruction179) +PUBLIC FN_PREFIX(CryptonightR_instruction180) +PUBLIC FN_PREFIX(CryptonightR_instruction181) +PUBLIC FN_PREFIX(CryptonightR_instruction182) +PUBLIC FN_PREFIX(CryptonightR_instruction183) +PUBLIC FN_PREFIX(CryptonightR_instruction184) +PUBLIC FN_PREFIX(CryptonightR_instruction185) +PUBLIC FN_PREFIX(CryptonightR_instruction186) +PUBLIC FN_PREFIX(CryptonightR_instruction187) +PUBLIC FN_PREFIX(CryptonightR_instruction188) +PUBLIC FN_PREFIX(CryptonightR_instruction189) +PUBLIC FN_PREFIX(CryptonightR_instruction190) +PUBLIC FN_PREFIX(CryptonightR_instruction191) +PUBLIC FN_PREFIX(CryptonightR_instruction192) +PUBLIC FN_PREFIX(CryptonightR_instruction193) +PUBLIC FN_PREFIX(CryptonightR_instruction194) +PUBLIC FN_PREFIX(CryptonightR_instruction195) +PUBLIC FN_PREFIX(CryptonightR_instruction196) +PUBLIC FN_PREFIX(CryptonightR_instruction197) +PUBLIC FN_PREFIX(CryptonightR_instruction198) +PUBLIC FN_PREFIX(CryptonightR_instruction199) +PUBLIC FN_PREFIX(CryptonightR_instruction200) +PUBLIC FN_PREFIX(CryptonightR_instruction201) +PUBLIC FN_PREFIX(CryptonightR_instruction202) +PUBLIC FN_PREFIX(CryptonightR_instruction203) +PUBLIC FN_PREFIX(CryptonightR_instruction204) +PUBLIC FN_PREFIX(CryptonightR_instruction205) +PUBLIC FN_PREFIX(CryptonightR_instruction206) +PUBLIC FN_PREFIX(CryptonightR_instruction207) +PUBLIC FN_PREFIX(CryptonightR_instruction208) +PUBLIC FN_PREFIX(CryptonightR_instruction209) +PUBLIC FN_PREFIX(CryptonightR_instruction210) +PUBLIC FN_PREFIX(CryptonightR_instruction211) +PUBLIC FN_PREFIX(CryptonightR_instruction212) +PUBLIC FN_PREFIX(CryptonightR_instruction213) +PUBLIC FN_PREFIX(CryptonightR_instruction214) +PUBLIC FN_PREFIX(CryptonightR_instruction215) +PUBLIC FN_PREFIX(CryptonightR_instruction216) +PUBLIC FN_PREFIX(CryptonightR_instruction217) +PUBLIC FN_PREFIX(CryptonightR_instruction218) +PUBLIC FN_PREFIX(CryptonightR_instruction219) +PUBLIC FN_PREFIX(CryptonightR_instruction220) +PUBLIC FN_PREFIX(CryptonightR_instruction221) +PUBLIC FN_PREFIX(CryptonightR_instruction222) +PUBLIC FN_PREFIX(CryptonightR_instruction223) +PUBLIC FN_PREFIX(CryptonightR_instruction224) +PUBLIC FN_PREFIX(CryptonightR_instruction225) +PUBLIC FN_PREFIX(CryptonightR_instruction226) +PUBLIC FN_PREFIX(CryptonightR_instruction227) +PUBLIC FN_PREFIX(CryptonightR_instruction228) +PUBLIC FN_PREFIX(CryptonightR_instruction229) +PUBLIC FN_PREFIX(CryptonightR_instruction230) +PUBLIC FN_PREFIX(CryptonightR_instruction231) +PUBLIC FN_PREFIX(CryptonightR_instruction232) +PUBLIC FN_PREFIX(CryptonightR_instruction233) +PUBLIC FN_PREFIX(CryptonightR_instruction234) +PUBLIC FN_PREFIX(CryptonightR_instruction235) +PUBLIC FN_PREFIX(CryptonightR_instruction236) +PUBLIC FN_PREFIX(CryptonightR_instruction237) +PUBLIC FN_PREFIX(CryptonightR_instruction238) +PUBLIC FN_PREFIX(CryptonightR_instruction239) +PUBLIC FN_PREFIX(CryptonightR_instruction240) +PUBLIC FN_PREFIX(CryptonightR_instruction241) +PUBLIC FN_PREFIX(CryptonightR_instruction242) +PUBLIC FN_PREFIX(CryptonightR_instruction243) +PUBLIC FN_PREFIX(CryptonightR_instruction244) +PUBLIC FN_PREFIX(CryptonightR_instruction245) +PUBLIC FN_PREFIX(CryptonightR_instruction246) +PUBLIC FN_PREFIX(CryptonightR_instruction247) +PUBLIC FN_PREFIX(CryptonightR_instruction248) +PUBLIC FN_PREFIX(CryptonightR_instruction249) +PUBLIC FN_PREFIX(CryptonightR_instruction250) +PUBLIC FN_PREFIX(CryptonightR_instruction251) +PUBLIC FN_PREFIX(CryptonightR_instruction252) +PUBLIC FN_PREFIX(CryptonightR_instruction253) +PUBLIC FN_PREFIX(CryptonightR_instruction254) +PUBLIC FN_PREFIX(CryptonightR_instruction255) +PUBLIC FN_PREFIX(CryptonightR_instruction256) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov0) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov1) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov2) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov3) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov4) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov5) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov6) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov7) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov8) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov9) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov10) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov11) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov12) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov13) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov14) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov15) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov16) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov17) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov18) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov19) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov20) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov21) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov22) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov23) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov24) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov25) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov26) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov27) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov28) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov29) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov30) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov31) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov32) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov33) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov34) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov35) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov36) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov37) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov38) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov39) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov40) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov41) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov42) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov43) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov44) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov45) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov46) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov47) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov48) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov49) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov50) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov51) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov52) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov53) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov54) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov55) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov56) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov57) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov58) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov59) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov60) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov61) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov62) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov63) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov64) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov65) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov66) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov67) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov68) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov69) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov70) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov71) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov72) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov73) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov74) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov75) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov76) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov77) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov78) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov79) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov80) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov81) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov82) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov83) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov84) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov85) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov86) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov87) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov88) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov89) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov90) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov91) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov92) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov93) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov94) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov95) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov96) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov97) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov98) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov99) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov100) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov101) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov102) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov103) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov104) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov105) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov106) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov107) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov108) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov109) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov110) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov111) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov112) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov113) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov114) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov115) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov116) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov117) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov118) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov119) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov120) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov121) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov122) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov123) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov124) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov125) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov126) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov127) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov128) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov129) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov130) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov131) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov132) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov133) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov134) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov135) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov136) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov137) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov138) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov139) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov140) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov141) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov142) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov143) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov144) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov145) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov146) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov147) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov148) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov149) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov150) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov151) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov152) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov153) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov154) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov155) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov156) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov157) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov158) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov159) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov160) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov161) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov162) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov163) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov164) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov165) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov166) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov167) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov168) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov169) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov170) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov171) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov172) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov173) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov174) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov175) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov176) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov177) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov178) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov179) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov180) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov181) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov182) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov183) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov184) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov185) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov186) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov187) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov188) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov189) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov190) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov191) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov192) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov193) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov194) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov195) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov196) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov197) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov198) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov199) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov200) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov201) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov202) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov203) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov204) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov205) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov206) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov207) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov208) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov209) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov210) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov211) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov212) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov213) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov214) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov215) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov216) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov217) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov218) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov219) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov220) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov221) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov222) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov223) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov224) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov225) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov226) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov227) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov228) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov229) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov230) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov231) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov232) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov233) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov234) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov235) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov236) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov237) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov238) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov239) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov240) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov241) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov242) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov243) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov244) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov245) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov246) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov247) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov248) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov249) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov250) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov251) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov252) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov253) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov254) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov255) +PUBLIC FN_PREFIX(CryptonightR_instruction_mov256) + +FN_PREFIX(CryptonightR_instruction0): + imul ebx, ebx +FN_PREFIX(CryptonightR_instruction1): + imul ebx, ebx +FN_PREFIX(CryptonightR_instruction2): + imul ebx, ebx +FN_PREFIX(CryptonightR_instruction3): + add ebx, r9d + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction4): + sub ebx, r9d +FN_PREFIX(CryptonightR_instruction5): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction6): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction7): + xor ebx, r9d +FN_PREFIX(CryptonightR_instruction8): + imul esi, ebx +FN_PREFIX(CryptonightR_instruction9): + imul esi, ebx +FN_PREFIX(CryptonightR_instruction10): + imul esi, ebx +FN_PREFIX(CryptonightR_instruction11): + add esi, ebx + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction12): + sub esi, ebx +FN_PREFIX(CryptonightR_instruction13): + ror esi, cl +FN_PREFIX(CryptonightR_instruction14): + rol esi, cl +FN_PREFIX(CryptonightR_instruction15): + xor esi, ebx +FN_PREFIX(CryptonightR_instruction16): + imul edi, ebx +FN_PREFIX(CryptonightR_instruction17): + imul edi, ebx +FN_PREFIX(CryptonightR_instruction18): + imul edi, ebx +FN_PREFIX(CryptonightR_instruction19): + add edi, ebx + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction20): + sub edi, ebx +FN_PREFIX(CryptonightR_instruction21): + ror edi, cl +FN_PREFIX(CryptonightR_instruction22): + rol edi, cl +FN_PREFIX(CryptonightR_instruction23): + xor edi, ebx +FN_PREFIX(CryptonightR_instruction24): + imul ebp, ebx +FN_PREFIX(CryptonightR_instruction25): + imul ebp, ebx +FN_PREFIX(CryptonightR_instruction26): + imul ebp, ebx +FN_PREFIX(CryptonightR_instruction27): + add ebp, ebx + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction28): + sub ebp, ebx +FN_PREFIX(CryptonightR_instruction29): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction30): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction31): + xor ebp, ebx +FN_PREFIX(CryptonightR_instruction32): + imul ebx, esi +FN_PREFIX(CryptonightR_instruction33): + imul ebx, esi +FN_PREFIX(CryptonightR_instruction34): + imul ebx, esi +FN_PREFIX(CryptonightR_instruction35): + add ebx, esi + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction36): + sub ebx, esi +FN_PREFIX(CryptonightR_instruction37): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction38): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction39): + xor ebx, esi +FN_PREFIX(CryptonightR_instruction40): + imul esi, esi +FN_PREFIX(CryptonightR_instruction41): + imul esi, esi +FN_PREFIX(CryptonightR_instruction42): + imul esi, esi +FN_PREFIX(CryptonightR_instruction43): + add esi, r9d + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction44): + sub esi, r9d +FN_PREFIX(CryptonightR_instruction45): + ror esi, cl +FN_PREFIX(CryptonightR_instruction46): + rol esi, cl +FN_PREFIX(CryptonightR_instruction47): + xor esi, r9d +FN_PREFIX(CryptonightR_instruction48): + imul edi, esi +FN_PREFIX(CryptonightR_instruction49): + imul edi, esi +FN_PREFIX(CryptonightR_instruction50): + imul edi, esi +FN_PREFIX(CryptonightR_instruction51): + add edi, esi + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction52): + sub edi, esi +FN_PREFIX(CryptonightR_instruction53): + ror edi, cl +FN_PREFIX(CryptonightR_instruction54): + rol edi, cl +FN_PREFIX(CryptonightR_instruction55): + xor edi, esi +FN_PREFIX(CryptonightR_instruction56): + imul ebp, esi +FN_PREFIX(CryptonightR_instruction57): + imul ebp, esi +FN_PREFIX(CryptonightR_instruction58): + imul ebp, esi +FN_PREFIX(CryptonightR_instruction59): + add ebp, esi + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction60): + sub ebp, esi +FN_PREFIX(CryptonightR_instruction61): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction62): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction63): + xor ebp, esi +FN_PREFIX(CryptonightR_instruction64): + imul ebx, edi +FN_PREFIX(CryptonightR_instruction65): + imul ebx, edi +FN_PREFIX(CryptonightR_instruction66): + imul ebx, edi +FN_PREFIX(CryptonightR_instruction67): + add ebx, edi + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction68): + sub ebx, edi +FN_PREFIX(CryptonightR_instruction69): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction70): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction71): + xor ebx, edi +FN_PREFIX(CryptonightR_instruction72): + imul esi, edi +FN_PREFIX(CryptonightR_instruction73): + imul esi, edi +FN_PREFIX(CryptonightR_instruction74): + imul esi, edi +FN_PREFIX(CryptonightR_instruction75): + add esi, edi + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction76): + sub esi, edi +FN_PREFIX(CryptonightR_instruction77): + ror esi, cl +FN_PREFIX(CryptonightR_instruction78): + rol esi, cl +FN_PREFIX(CryptonightR_instruction79): + xor esi, edi +FN_PREFIX(CryptonightR_instruction80): + imul edi, edi +FN_PREFIX(CryptonightR_instruction81): + imul edi, edi +FN_PREFIX(CryptonightR_instruction82): + imul edi, edi +FN_PREFIX(CryptonightR_instruction83): + add edi, r9d + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction84): + sub edi, r9d +FN_PREFIX(CryptonightR_instruction85): + ror edi, cl +FN_PREFIX(CryptonightR_instruction86): + rol edi, cl +FN_PREFIX(CryptonightR_instruction87): + xor edi, r9d +FN_PREFIX(CryptonightR_instruction88): + imul ebp, edi +FN_PREFIX(CryptonightR_instruction89): + imul ebp, edi +FN_PREFIX(CryptonightR_instruction90): + imul ebp, edi +FN_PREFIX(CryptonightR_instruction91): + add ebp, edi + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction92): + sub ebp, edi +FN_PREFIX(CryptonightR_instruction93): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction94): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction95): + xor ebp, edi +FN_PREFIX(CryptonightR_instruction96): + imul ebx, ebp +FN_PREFIX(CryptonightR_instruction97): + imul ebx, ebp +FN_PREFIX(CryptonightR_instruction98): + imul ebx, ebp +FN_PREFIX(CryptonightR_instruction99): + add ebx, ebp + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction100): + sub ebx, ebp +FN_PREFIX(CryptonightR_instruction101): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction102): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction103): + xor ebx, ebp +FN_PREFIX(CryptonightR_instruction104): + imul esi, ebp +FN_PREFIX(CryptonightR_instruction105): + imul esi, ebp +FN_PREFIX(CryptonightR_instruction106): + imul esi, ebp +FN_PREFIX(CryptonightR_instruction107): + add esi, ebp + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction108): + sub esi, ebp +FN_PREFIX(CryptonightR_instruction109): + ror esi, cl +FN_PREFIX(CryptonightR_instruction110): + rol esi, cl +FN_PREFIX(CryptonightR_instruction111): + xor esi, ebp +FN_PREFIX(CryptonightR_instruction112): + imul edi, ebp +FN_PREFIX(CryptonightR_instruction113): + imul edi, ebp +FN_PREFIX(CryptonightR_instruction114): + imul edi, ebp +FN_PREFIX(CryptonightR_instruction115): + add edi, ebp + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction116): + sub edi, ebp +FN_PREFIX(CryptonightR_instruction117): + ror edi, cl +FN_PREFIX(CryptonightR_instruction118): + rol edi, cl +FN_PREFIX(CryptonightR_instruction119): + xor edi, ebp +FN_PREFIX(CryptonightR_instruction120): + imul ebp, ebp +FN_PREFIX(CryptonightR_instruction121): + imul ebp, ebp +FN_PREFIX(CryptonightR_instruction122): + imul ebp, ebp +FN_PREFIX(CryptonightR_instruction123): + add ebp, r9d + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction124): + sub ebp, r9d +FN_PREFIX(CryptonightR_instruction125): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction126): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction127): + xor ebp, r9d +FN_PREFIX(CryptonightR_instruction128): + imul ebx, esp +FN_PREFIX(CryptonightR_instruction129): + imul ebx, esp +FN_PREFIX(CryptonightR_instruction130): + imul ebx, esp +FN_PREFIX(CryptonightR_instruction131): + add ebx, esp + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction132): + sub ebx, esp +FN_PREFIX(CryptonightR_instruction133): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction134): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction135): + xor ebx, esp +FN_PREFIX(CryptonightR_instruction136): + imul esi, esp +FN_PREFIX(CryptonightR_instruction137): + imul esi, esp +FN_PREFIX(CryptonightR_instruction138): + imul esi, esp +FN_PREFIX(CryptonightR_instruction139): + add esi, esp + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction140): + sub esi, esp +FN_PREFIX(CryptonightR_instruction141): + ror esi, cl +FN_PREFIX(CryptonightR_instruction142): + rol esi, cl +FN_PREFIX(CryptonightR_instruction143): + xor esi, esp +FN_PREFIX(CryptonightR_instruction144): + imul edi, esp +FN_PREFIX(CryptonightR_instruction145): + imul edi, esp +FN_PREFIX(CryptonightR_instruction146): + imul edi, esp +FN_PREFIX(CryptonightR_instruction147): + add edi, esp + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction148): + sub edi, esp +FN_PREFIX(CryptonightR_instruction149): + ror edi, cl +FN_PREFIX(CryptonightR_instruction150): + rol edi, cl +FN_PREFIX(CryptonightR_instruction151): + xor edi, esp +FN_PREFIX(CryptonightR_instruction152): + imul ebp, esp +FN_PREFIX(CryptonightR_instruction153): + imul ebp, esp +FN_PREFIX(CryptonightR_instruction154): + imul ebp, esp +FN_PREFIX(CryptonightR_instruction155): + add ebp, esp + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction156): + sub ebp, esp +FN_PREFIX(CryptonightR_instruction157): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction158): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction159): + xor ebp, esp +FN_PREFIX(CryptonightR_instruction160): + imul ebx, r15d +FN_PREFIX(CryptonightR_instruction161): + imul ebx, r15d +FN_PREFIX(CryptonightR_instruction162): + imul ebx, r15d +FN_PREFIX(CryptonightR_instruction163): + add ebx, r15d + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction164): + sub ebx, r15d +FN_PREFIX(CryptonightR_instruction165): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction166): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction167): + xor ebx, r15d +FN_PREFIX(CryptonightR_instruction168): + imul esi, r15d +FN_PREFIX(CryptonightR_instruction169): + imul esi, r15d +FN_PREFIX(CryptonightR_instruction170): + imul esi, r15d +FN_PREFIX(CryptonightR_instruction171): + add esi, r15d + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction172): + sub esi, r15d +FN_PREFIX(CryptonightR_instruction173): + ror esi, cl +FN_PREFIX(CryptonightR_instruction174): + rol esi, cl +FN_PREFIX(CryptonightR_instruction175): + xor esi, r15d +FN_PREFIX(CryptonightR_instruction176): + imul edi, r15d +FN_PREFIX(CryptonightR_instruction177): + imul edi, r15d +FN_PREFIX(CryptonightR_instruction178): + imul edi, r15d +FN_PREFIX(CryptonightR_instruction179): + add edi, r15d + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction180): + sub edi, r15d +FN_PREFIX(CryptonightR_instruction181): + ror edi, cl +FN_PREFIX(CryptonightR_instruction182): + rol edi, cl +FN_PREFIX(CryptonightR_instruction183): + xor edi, r15d +FN_PREFIX(CryptonightR_instruction184): + imul ebp, r15d +FN_PREFIX(CryptonightR_instruction185): + imul ebp, r15d +FN_PREFIX(CryptonightR_instruction186): + imul ebp, r15d +FN_PREFIX(CryptonightR_instruction187): + add ebp, r15d + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction188): + sub ebp, r15d +FN_PREFIX(CryptonightR_instruction189): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction190): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction191): + xor ebp, r15d +FN_PREFIX(CryptonightR_instruction192): + imul ebx, eax +FN_PREFIX(CryptonightR_instruction193): + imul ebx, eax +FN_PREFIX(CryptonightR_instruction194): + imul ebx, eax +FN_PREFIX(CryptonightR_instruction195): + add ebx, eax + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction196): + sub ebx, eax +FN_PREFIX(CryptonightR_instruction197): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction198): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction199): + xor ebx, eax +FN_PREFIX(CryptonightR_instruction200): + imul esi, eax +FN_PREFIX(CryptonightR_instruction201): + imul esi, eax +FN_PREFIX(CryptonightR_instruction202): + imul esi, eax +FN_PREFIX(CryptonightR_instruction203): + add esi, eax + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction204): + sub esi, eax +FN_PREFIX(CryptonightR_instruction205): + ror esi, cl +FN_PREFIX(CryptonightR_instruction206): + rol esi, cl +FN_PREFIX(CryptonightR_instruction207): + xor esi, eax +FN_PREFIX(CryptonightR_instruction208): + imul edi, eax +FN_PREFIX(CryptonightR_instruction209): + imul edi, eax +FN_PREFIX(CryptonightR_instruction210): + imul edi, eax +FN_PREFIX(CryptonightR_instruction211): + add edi, eax + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction212): + sub edi, eax +FN_PREFIX(CryptonightR_instruction213): + ror edi, cl +FN_PREFIX(CryptonightR_instruction214): + rol edi, cl +FN_PREFIX(CryptonightR_instruction215): + xor edi, eax +FN_PREFIX(CryptonightR_instruction216): + imul ebp, eax +FN_PREFIX(CryptonightR_instruction217): + imul ebp, eax +FN_PREFIX(CryptonightR_instruction218): + imul ebp, eax +FN_PREFIX(CryptonightR_instruction219): + add ebp, eax + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction220): + sub ebp, eax +FN_PREFIX(CryptonightR_instruction221): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction222): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction223): + xor ebp, eax +FN_PREFIX(CryptonightR_instruction224): + imul ebx, edx +FN_PREFIX(CryptonightR_instruction225): + imul ebx, edx +FN_PREFIX(CryptonightR_instruction226): + imul ebx, edx +FN_PREFIX(CryptonightR_instruction227): + add ebx, edx + add ebx, 2147483647 +FN_PREFIX(CryptonightR_instruction228): + sub ebx, edx +FN_PREFIX(CryptonightR_instruction229): + ror ebx, cl +FN_PREFIX(CryptonightR_instruction230): + rol ebx, cl +FN_PREFIX(CryptonightR_instruction231): + xor ebx, edx +FN_PREFIX(CryptonightR_instruction232): + imul esi, edx +FN_PREFIX(CryptonightR_instruction233): + imul esi, edx +FN_PREFIX(CryptonightR_instruction234): + imul esi, edx +FN_PREFIX(CryptonightR_instruction235): + add esi, edx + add esi, 2147483647 +FN_PREFIX(CryptonightR_instruction236): + sub esi, edx +FN_PREFIX(CryptonightR_instruction237): + ror esi, cl +FN_PREFIX(CryptonightR_instruction238): + rol esi, cl +FN_PREFIX(CryptonightR_instruction239): + xor esi, edx +FN_PREFIX(CryptonightR_instruction240): + imul edi, edx +FN_PREFIX(CryptonightR_instruction241): + imul edi, edx +FN_PREFIX(CryptonightR_instruction242): + imul edi, edx +FN_PREFIX(CryptonightR_instruction243): + add edi, edx + add edi, 2147483647 +FN_PREFIX(CryptonightR_instruction244): + sub edi, edx +FN_PREFIX(CryptonightR_instruction245): + ror edi, cl +FN_PREFIX(CryptonightR_instruction246): + rol edi, cl +FN_PREFIX(CryptonightR_instruction247): + xor edi, edx +FN_PREFIX(CryptonightR_instruction248): + imul ebp, edx +FN_PREFIX(CryptonightR_instruction249): + imul ebp, edx +FN_PREFIX(CryptonightR_instruction250): + imul ebp, edx +FN_PREFIX(CryptonightR_instruction251): + add ebp, edx + add ebp, 2147483647 +FN_PREFIX(CryptonightR_instruction252): + sub ebp, edx +FN_PREFIX(CryptonightR_instruction253): + ror ebp, cl +FN_PREFIX(CryptonightR_instruction254): + rol ebp, cl +FN_PREFIX(CryptonightR_instruction255): + xor ebp, edx +FN_PREFIX(CryptonightR_instruction256): + imul ebx, ebx +FN_PREFIX(CryptonightR_instruction_mov0): + +FN_PREFIX(CryptonightR_instruction_mov1): + +FN_PREFIX(CryptonightR_instruction_mov2): + +FN_PREFIX(CryptonightR_instruction_mov3): + +FN_PREFIX(CryptonightR_instruction_mov4): + +FN_PREFIX(CryptonightR_instruction_mov5): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov6): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov7): + +FN_PREFIX(CryptonightR_instruction_mov8): + +FN_PREFIX(CryptonightR_instruction_mov9): + +FN_PREFIX(CryptonightR_instruction_mov10): + +FN_PREFIX(CryptonightR_instruction_mov11): + +FN_PREFIX(CryptonightR_instruction_mov12): + +FN_PREFIX(CryptonightR_instruction_mov13): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov14): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov15): + +FN_PREFIX(CryptonightR_instruction_mov16): + +FN_PREFIX(CryptonightR_instruction_mov17): + +FN_PREFIX(CryptonightR_instruction_mov18): + +FN_PREFIX(CryptonightR_instruction_mov19): + +FN_PREFIX(CryptonightR_instruction_mov20): + +FN_PREFIX(CryptonightR_instruction_mov21): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov22): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov23): + +FN_PREFIX(CryptonightR_instruction_mov24): + +FN_PREFIX(CryptonightR_instruction_mov25): + +FN_PREFIX(CryptonightR_instruction_mov26): + +FN_PREFIX(CryptonightR_instruction_mov27): + +FN_PREFIX(CryptonightR_instruction_mov28): + +FN_PREFIX(CryptonightR_instruction_mov29): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov30): + mov ecx, ebx +FN_PREFIX(CryptonightR_instruction_mov31): + +FN_PREFIX(CryptonightR_instruction_mov32): + +FN_PREFIX(CryptonightR_instruction_mov33): + +FN_PREFIX(CryptonightR_instruction_mov34): + +FN_PREFIX(CryptonightR_instruction_mov35): + +FN_PREFIX(CryptonightR_instruction_mov36): + +FN_PREFIX(CryptonightR_instruction_mov37): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov38): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov39): + +FN_PREFIX(CryptonightR_instruction_mov40): + +FN_PREFIX(CryptonightR_instruction_mov41): + +FN_PREFIX(CryptonightR_instruction_mov42): + +FN_PREFIX(CryptonightR_instruction_mov43): + +FN_PREFIX(CryptonightR_instruction_mov44): + +FN_PREFIX(CryptonightR_instruction_mov45): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov46): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov47): + +FN_PREFIX(CryptonightR_instruction_mov48): + +FN_PREFIX(CryptonightR_instruction_mov49): + +FN_PREFIX(CryptonightR_instruction_mov50): + +FN_PREFIX(CryptonightR_instruction_mov51): + +FN_PREFIX(CryptonightR_instruction_mov52): + +FN_PREFIX(CryptonightR_instruction_mov53): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov54): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov55): + +FN_PREFIX(CryptonightR_instruction_mov56): + +FN_PREFIX(CryptonightR_instruction_mov57): + +FN_PREFIX(CryptonightR_instruction_mov58): + +FN_PREFIX(CryptonightR_instruction_mov59): + +FN_PREFIX(CryptonightR_instruction_mov60): + +FN_PREFIX(CryptonightR_instruction_mov61): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov62): + mov ecx, esi +FN_PREFIX(CryptonightR_instruction_mov63): + +FN_PREFIX(CryptonightR_instruction_mov64): + +FN_PREFIX(CryptonightR_instruction_mov65): + +FN_PREFIX(CryptonightR_instruction_mov66): + +FN_PREFIX(CryptonightR_instruction_mov67): + +FN_PREFIX(CryptonightR_instruction_mov68): + +FN_PREFIX(CryptonightR_instruction_mov69): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov70): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov71): + +FN_PREFIX(CryptonightR_instruction_mov72): + +FN_PREFIX(CryptonightR_instruction_mov73): + +FN_PREFIX(CryptonightR_instruction_mov74): + +FN_PREFIX(CryptonightR_instruction_mov75): + +FN_PREFIX(CryptonightR_instruction_mov76): + +FN_PREFIX(CryptonightR_instruction_mov77): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov78): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov79): + +FN_PREFIX(CryptonightR_instruction_mov80): + +FN_PREFIX(CryptonightR_instruction_mov81): + +FN_PREFIX(CryptonightR_instruction_mov82): + +FN_PREFIX(CryptonightR_instruction_mov83): + +FN_PREFIX(CryptonightR_instruction_mov84): + +FN_PREFIX(CryptonightR_instruction_mov85): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov86): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov87): + +FN_PREFIX(CryptonightR_instruction_mov88): + +FN_PREFIX(CryptonightR_instruction_mov89): + +FN_PREFIX(CryptonightR_instruction_mov90): + +FN_PREFIX(CryptonightR_instruction_mov91): + +FN_PREFIX(CryptonightR_instruction_mov92): + +FN_PREFIX(CryptonightR_instruction_mov93): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov94): + mov ecx, edi +FN_PREFIX(CryptonightR_instruction_mov95): + +FN_PREFIX(CryptonightR_instruction_mov96): + +FN_PREFIX(CryptonightR_instruction_mov97): + +FN_PREFIX(CryptonightR_instruction_mov98): + +FN_PREFIX(CryptonightR_instruction_mov99): + +FN_PREFIX(CryptonightR_instruction_mov100): + +FN_PREFIX(CryptonightR_instruction_mov101): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov102): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov103): + +FN_PREFIX(CryptonightR_instruction_mov104): + +FN_PREFIX(CryptonightR_instruction_mov105): + +FN_PREFIX(CryptonightR_instruction_mov106): + +FN_PREFIX(CryptonightR_instruction_mov107): + +FN_PREFIX(CryptonightR_instruction_mov108): + +FN_PREFIX(CryptonightR_instruction_mov109): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov110): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov111): + +FN_PREFIX(CryptonightR_instruction_mov112): + +FN_PREFIX(CryptonightR_instruction_mov113): + +FN_PREFIX(CryptonightR_instruction_mov114): + +FN_PREFIX(CryptonightR_instruction_mov115): + +FN_PREFIX(CryptonightR_instruction_mov116): + +FN_PREFIX(CryptonightR_instruction_mov117): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov118): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov119): + +FN_PREFIX(CryptonightR_instruction_mov120): + +FN_PREFIX(CryptonightR_instruction_mov121): + +FN_PREFIX(CryptonightR_instruction_mov122): + +FN_PREFIX(CryptonightR_instruction_mov123): + +FN_PREFIX(CryptonightR_instruction_mov124): + +FN_PREFIX(CryptonightR_instruction_mov125): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov126): + mov ecx, ebp +FN_PREFIX(CryptonightR_instruction_mov127): + +FN_PREFIX(CryptonightR_instruction_mov128): + +FN_PREFIX(CryptonightR_instruction_mov129): + +FN_PREFIX(CryptonightR_instruction_mov130): + +FN_PREFIX(CryptonightR_instruction_mov131): + +FN_PREFIX(CryptonightR_instruction_mov132): + +FN_PREFIX(CryptonightR_instruction_mov133): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov134): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov135): + +FN_PREFIX(CryptonightR_instruction_mov136): + +FN_PREFIX(CryptonightR_instruction_mov137): + +FN_PREFIX(CryptonightR_instruction_mov138): + +FN_PREFIX(CryptonightR_instruction_mov139): + +FN_PREFIX(CryptonightR_instruction_mov140): + +FN_PREFIX(CryptonightR_instruction_mov141): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov142): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov143): + +FN_PREFIX(CryptonightR_instruction_mov144): + +FN_PREFIX(CryptonightR_instruction_mov145): + +FN_PREFIX(CryptonightR_instruction_mov146): + +FN_PREFIX(CryptonightR_instruction_mov147): + +FN_PREFIX(CryptonightR_instruction_mov148): + +FN_PREFIX(CryptonightR_instruction_mov149): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov150): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov151): + +FN_PREFIX(CryptonightR_instruction_mov152): + +FN_PREFIX(CryptonightR_instruction_mov153): + +FN_PREFIX(CryptonightR_instruction_mov154): + +FN_PREFIX(CryptonightR_instruction_mov155): + +FN_PREFIX(CryptonightR_instruction_mov156): + +FN_PREFIX(CryptonightR_instruction_mov157): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov158): + mov ecx, esp +FN_PREFIX(CryptonightR_instruction_mov159): + +FN_PREFIX(CryptonightR_instruction_mov160): + +FN_PREFIX(CryptonightR_instruction_mov161): + +FN_PREFIX(CryptonightR_instruction_mov162): + +FN_PREFIX(CryptonightR_instruction_mov163): + +FN_PREFIX(CryptonightR_instruction_mov164): + +FN_PREFIX(CryptonightR_instruction_mov165): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov166): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov167): + +FN_PREFIX(CryptonightR_instruction_mov168): + +FN_PREFIX(CryptonightR_instruction_mov169): + +FN_PREFIX(CryptonightR_instruction_mov170): + +FN_PREFIX(CryptonightR_instruction_mov171): + +FN_PREFIX(CryptonightR_instruction_mov172): + +FN_PREFIX(CryptonightR_instruction_mov173): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov174): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov175): + +FN_PREFIX(CryptonightR_instruction_mov176): + +FN_PREFIX(CryptonightR_instruction_mov177): + +FN_PREFIX(CryptonightR_instruction_mov178): + +FN_PREFIX(CryptonightR_instruction_mov179): + +FN_PREFIX(CryptonightR_instruction_mov180): + +FN_PREFIX(CryptonightR_instruction_mov181): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov182): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov183): + +FN_PREFIX(CryptonightR_instruction_mov184): + +FN_PREFIX(CryptonightR_instruction_mov185): + +FN_PREFIX(CryptonightR_instruction_mov186): + +FN_PREFIX(CryptonightR_instruction_mov187): + +FN_PREFIX(CryptonightR_instruction_mov188): + +FN_PREFIX(CryptonightR_instruction_mov189): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov190): + mov ecx, r15d +FN_PREFIX(CryptonightR_instruction_mov191): + +FN_PREFIX(CryptonightR_instruction_mov192): + +FN_PREFIX(CryptonightR_instruction_mov193): + +FN_PREFIX(CryptonightR_instruction_mov194): + +FN_PREFIX(CryptonightR_instruction_mov195): + +FN_PREFIX(CryptonightR_instruction_mov196): + +FN_PREFIX(CryptonightR_instruction_mov197): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov198): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov199): + +FN_PREFIX(CryptonightR_instruction_mov200): + +FN_PREFIX(CryptonightR_instruction_mov201): + +FN_PREFIX(CryptonightR_instruction_mov202): + +FN_PREFIX(CryptonightR_instruction_mov203): + +FN_PREFIX(CryptonightR_instruction_mov204): + +FN_PREFIX(CryptonightR_instruction_mov205): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov206): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov207): + +FN_PREFIX(CryptonightR_instruction_mov208): + +FN_PREFIX(CryptonightR_instruction_mov209): + +FN_PREFIX(CryptonightR_instruction_mov210): + +FN_PREFIX(CryptonightR_instruction_mov211): + +FN_PREFIX(CryptonightR_instruction_mov212): + +FN_PREFIX(CryptonightR_instruction_mov213): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov214): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov215): + +FN_PREFIX(CryptonightR_instruction_mov216): + +FN_PREFIX(CryptonightR_instruction_mov217): + +FN_PREFIX(CryptonightR_instruction_mov218): + +FN_PREFIX(CryptonightR_instruction_mov219): + +FN_PREFIX(CryptonightR_instruction_mov220): + +FN_PREFIX(CryptonightR_instruction_mov221): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov222): + mov ecx, eax +FN_PREFIX(CryptonightR_instruction_mov223): + +FN_PREFIX(CryptonightR_instruction_mov224): + +FN_PREFIX(CryptonightR_instruction_mov225): + +FN_PREFIX(CryptonightR_instruction_mov226): + +FN_PREFIX(CryptonightR_instruction_mov227): + +FN_PREFIX(CryptonightR_instruction_mov228): + +FN_PREFIX(CryptonightR_instruction_mov229): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov230): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov231): + +FN_PREFIX(CryptonightR_instruction_mov232): + +FN_PREFIX(CryptonightR_instruction_mov233): + +FN_PREFIX(CryptonightR_instruction_mov234): + +FN_PREFIX(CryptonightR_instruction_mov235): + +FN_PREFIX(CryptonightR_instruction_mov236): + +FN_PREFIX(CryptonightR_instruction_mov237): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov238): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov239): + +FN_PREFIX(CryptonightR_instruction_mov240): + +FN_PREFIX(CryptonightR_instruction_mov241): + +FN_PREFIX(CryptonightR_instruction_mov242): + +FN_PREFIX(CryptonightR_instruction_mov243): + +FN_PREFIX(CryptonightR_instruction_mov244): + +FN_PREFIX(CryptonightR_instruction_mov245): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov246): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov247): + +FN_PREFIX(CryptonightR_instruction_mov248): + +FN_PREFIX(CryptonightR_instruction_mov249): + +FN_PREFIX(CryptonightR_instruction_mov250): + +FN_PREFIX(CryptonightR_instruction_mov251): + +FN_PREFIX(CryptonightR_instruction_mov252): + +FN_PREFIX(CryptonightR_instruction_mov253): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov254): + mov ecx, edx +FN_PREFIX(CryptonightR_instruction_mov255): + +FN_PREFIX(CryptonightR_instruction_mov256): diff --git a/src/crypto/CryptonightR_template.h b/src/crypto/CryptonightR_template.h new file mode 100644 index 000000000..57eb92ebe --- /dev/null +++ b/src/crypto/CryptonightR_template.h @@ -0,0 +1,1039 @@ +#ifndef CRYPTONIGHTR_TEMPLATE_H +#define CRYPTONIGHTR_TEMPLATE_H + +void CryptonightR_instruction0(void); +void CryptonightR_instruction1(void); +void CryptonightR_instruction2(void); +void CryptonightR_instruction3(void); +void CryptonightR_instruction4(void); +void CryptonightR_instruction5(void); +void CryptonightR_instruction6(void); +void CryptonightR_instruction7(void); +void CryptonightR_instruction8(void); +void CryptonightR_instruction9(void); +void CryptonightR_instruction10(void); +void CryptonightR_instruction11(void); +void CryptonightR_instruction12(void); +void CryptonightR_instruction13(void); +void CryptonightR_instruction14(void); +void CryptonightR_instruction15(void); +void CryptonightR_instruction16(void); +void CryptonightR_instruction17(void); +void CryptonightR_instruction18(void); +void CryptonightR_instruction19(void); +void CryptonightR_instruction20(void); +void CryptonightR_instruction21(void); +void CryptonightR_instruction22(void); +void CryptonightR_instruction23(void); +void CryptonightR_instruction24(void); +void CryptonightR_instruction25(void); +void CryptonightR_instruction26(void); +void CryptonightR_instruction27(void); +void CryptonightR_instruction28(void); +void CryptonightR_instruction29(void); +void CryptonightR_instruction30(void); +void CryptonightR_instruction31(void); +void CryptonightR_instruction32(void); +void CryptonightR_instruction33(void); +void CryptonightR_instruction34(void); +void CryptonightR_instruction35(void); +void CryptonightR_instruction36(void); +void CryptonightR_instruction37(void); +void CryptonightR_instruction38(void); +void CryptonightR_instruction39(void); +void CryptonightR_instruction40(void); +void CryptonightR_instruction41(void); +void CryptonightR_instruction42(void); +void CryptonightR_instruction43(void); +void CryptonightR_instruction44(void); +void CryptonightR_instruction45(void); +void CryptonightR_instruction46(void); +void CryptonightR_instruction47(void); +void CryptonightR_instruction48(void); +void CryptonightR_instruction49(void); +void CryptonightR_instruction50(void); +void CryptonightR_instruction51(void); +void CryptonightR_instruction52(void); +void CryptonightR_instruction53(void); +void CryptonightR_instruction54(void); +void CryptonightR_instruction55(void); +void CryptonightR_instruction56(void); +void CryptonightR_instruction57(void); +void CryptonightR_instruction58(void); +void CryptonightR_instruction59(void); +void CryptonightR_instruction60(void); +void CryptonightR_instruction61(void); +void CryptonightR_instruction62(void); +void CryptonightR_instruction63(void); +void CryptonightR_instruction64(void); +void CryptonightR_instruction65(void); +void CryptonightR_instruction66(void); +void CryptonightR_instruction67(void); +void CryptonightR_instruction68(void); +void CryptonightR_instruction69(void); +void CryptonightR_instruction70(void); +void CryptonightR_instruction71(void); +void CryptonightR_instruction72(void); +void CryptonightR_instruction73(void); +void CryptonightR_instruction74(void); +void CryptonightR_instruction75(void); +void CryptonightR_instruction76(void); +void CryptonightR_instruction77(void); +void CryptonightR_instruction78(void); +void CryptonightR_instruction79(void); +void CryptonightR_instruction80(void); +void CryptonightR_instruction81(void); +void CryptonightR_instruction82(void); +void CryptonightR_instruction83(void); +void CryptonightR_instruction84(void); +void CryptonightR_instruction85(void); +void CryptonightR_instruction86(void); +void CryptonightR_instruction87(void); +void CryptonightR_instruction88(void); +void CryptonightR_instruction89(void); +void CryptonightR_instruction90(void); +void CryptonightR_instruction91(void); +void CryptonightR_instruction92(void); +void CryptonightR_instruction93(void); +void CryptonightR_instruction94(void); +void CryptonightR_instruction95(void); +void CryptonightR_instruction96(void); +void CryptonightR_instruction97(void); +void CryptonightR_instruction98(void); +void CryptonightR_instruction99(void); +void CryptonightR_instruction100(void); +void CryptonightR_instruction101(void); +void CryptonightR_instruction102(void); +void CryptonightR_instruction103(void); +void CryptonightR_instruction104(void); +void CryptonightR_instruction105(void); +void CryptonightR_instruction106(void); +void CryptonightR_instruction107(void); +void CryptonightR_instruction108(void); +void CryptonightR_instruction109(void); +void CryptonightR_instruction110(void); +void CryptonightR_instruction111(void); +void CryptonightR_instruction112(void); +void CryptonightR_instruction113(void); +void CryptonightR_instruction114(void); +void CryptonightR_instruction115(void); +void CryptonightR_instruction116(void); +void CryptonightR_instruction117(void); +void CryptonightR_instruction118(void); +void CryptonightR_instruction119(void); +void CryptonightR_instruction120(void); +void CryptonightR_instruction121(void); +void CryptonightR_instruction122(void); +void CryptonightR_instruction123(void); +void CryptonightR_instruction124(void); +void CryptonightR_instruction125(void); +void CryptonightR_instruction126(void); +void CryptonightR_instruction127(void); +void CryptonightR_instruction128(void); +void CryptonightR_instruction129(void); +void CryptonightR_instruction130(void); +void CryptonightR_instruction131(void); +void CryptonightR_instruction132(void); +void CryptonightR_instruction133(void); +void CryptonightR_instruction134(void); +void CryptonightR_instruction135(void); +void CryptonightR_instruction136(void); +void CryptonightR_instruction137(void); +void CryptonightR_instruction138(void); +void CryptonightR_instruction139(void); +void CryptonightR_instruction140(void); +void CryptonightR_instruction141(void); +void CryptonightR_instruction142(void); +void CryptonightR_instruction143(void); +void CryptonightR_instruction144(void); +void CryptonightR_instruction145(void); +void CryptonightR_instruction146(void); +void CryptonightR_instruction147(void); +void CryptonightR_instruction148(void); +void CryptonightR_instruction149(void); +void CryptonightR_instruction150(void); +void CryptonightR_instruction151(void); +void CryptonightR_instruction152(void); +void CryptonightR_instruction153(void); +void CryptonightR_instruction154(void); +void CryptonightR_instruction155(void); +void CryptonightR_instruction156(void); +void CryptonightR_instruction157(void); +void CryptonightR_instruction158(void); +void CryptonightR_instruction159(void); +void CryptonightR_instruction160(void); +void CryptonightR_instruction161(void); +void CryptonightR_instruction162(void); +void CryptonightR_instruction163(void); +void CryptonightR_instruction164(void); +void CryptonightR_instruction165(void); +void CryptonightR_instruction166(void); +void CryptonightR_instruction167(void); +void CryptonightR_instruction168(void); +void CryptonightR_instruction169(void); +void CryptonightR_instruction170(void); +void CryptonightR_instruction171(void); +void CryptonightR_instruction172(void); +void CryptonightR_instruction173(void); +void CryptonightR_instruction174(void); +void CryptonightR_instruction175(void); +void CryptonightR_instruction176(void); +void CryptonightR_instruction177(void); +void CryptonightR_instruction178(void); +void CryptonightR_instruction179(void); +void CryptonightR_instruction180(void); +void CryptonightR_instruction181(void); +void CryptonightR_instruction182(void); +void CryptonightR_instruction183(void); +void CryptonightR_instruction184(void); +void CryptonightR_instruction185(void); +void CryptonightR_instruction186(void); +void CryptonightR_instruction187(void); +void CryptonightR_instruction188(void); +void CryptonightR_instruction189(void); +void CryptonightR_instruction190(void); +void CryptonightR_instruction191(void); +void CryptonightR_instruction192(void); +void CryptonightR_instruction193(void); +void CryptonightR_instruction194(void); +void CryptonightR_instruction195(void); +void CryptonightR_instruction196(void); +void CryptonightR_instruction197(void); +void CryptonightR_instruction198(void); +void CryptonightR_instruction199(void); +void CryptonightR_instruction200(void); +void CryptonightR_instruction201(void); +void CryptonightR_instruction202(void); +void CryptonightR_instruction203(void); +void CryptonightR_instruction204(void); +void CryptonightR_instruction205(void); +void CryptonightR_instruction206(void); +void CryptonightR_instruction207(void); +void CryptonightR_instruction208(void); +void CryptonightR_instruction209(void); +void CryptonightR_instruction210(void); +void CryptonightR_instruction211(void); +void CryptonightR_instruction212(void); +void CryptonightR_instruction213(void); +void CryptonightR_instruction214(void); +void CryptonightR_instruction215(void); +void CryptonightR_instruction216(void); +void CryptonightR_instruction217(void); +void CryptonightR_instruction218(void); +void CryptonightR_instruction219(void); +void CryptonightR_instruction220(void); +void CryptonightR_instruction221(void); +void CryptonightR_instruction222(void); +void CryptonightR_instruction223(void); +void CryptonightR_instruction224(void); +void CryptonightR_instruction225(void); +void CryptonightR_instruction226(void); +void CryptonightR_instruction227(void); +void CryptonightR_instruction228(void); +void CryptonightR_instruction229(void); +void CryptonightR_instruction230(void); +void CryptonightR_instruction231(void); +void CryptonightR_instruction232(void); +void CryptonightR_instruction233(void); +void CryptonightR_instruction234(void); +void CryptonightR_instruction235(void); +void CryptonightR_instruction236(void); +void CryptonightR_instruction237(void); +void CryptonightR_instruction238(void); +void CryptonightR_instruction239(void); +void CryptonightR_instruction240(void); +void CryptonightR_instruction241(void); +void CryptonightR_instruction242(void); +void CryptonightR_instruction243(void); +void CryptonightR_instruction244(void); +void CryptonightR_instruction245(void); +void CryptonightR_instruction246(void); +void CryptonightR_instruction247(void); +void CryptonightR_instruction248(void); +void CryptonightR_instruction249(void); +void CryptonightR_instruction250(void); +void CryptonightR_instruction251(void); +void CryptonightR_instruction252(void); +void CryptonightR_instruction253(void); +void CryptonightR_instruction254(void); +void CryptonightR_instruction255(void); +void CryptonightR_instruction256(void); +void CryptonightR_instruction_mov0(void); +void CryptonightR_instruction_mov1(void); +void CryptonightR_instruction_mov2(void); +void CryptonightR_instruction_mov3(void); +void CryptonightR_instruction_mov4(void); +void CryptonightR_instruction_mov5(void); +void CryptonightR_instruction_mov6(void); +void CryptonightR_instruction_mov7(void); +void CryptonightR_instruction_mov8(void); +void CryptonightR_instruction_mov9(void); +void CryptonightR_instruction_mov10(void); +void CryptonightR_instruction_mov11(void); +void CryptonightR_instruction_mov12(void); +void CryptonightR_instruction_mov13(void); +void CryptonightR_instruction_mov14(void); +void CryptonightR_instruction_mov15(void); +void CryptonightR_instruction_mov16(void); +void CryptonightR_instruction_mov17(void); +void CryptonightR_instruction_mov18(void); +void CryptonightR_instruction_mov19(void); +void CryptonightR_instruction_mov20(void); +void CryptonightR_instruction_mov21(void); +void CryptonightR_instruction_mov22(void); +void CryptonightR_instruction_mov23(void); +void CryptonightR_instruction_mov24(void); +void CryptonightR_instruction_mov25(void); +void CryptonightR_instruction_mov26(void); +void CryptonightR_instruction_mov27(void); +void CryptonightR_instruction_mov28(void); +void CryptonightR_instruction_mov29(void); +void CryptonightR_instruction_mov30(void); +void CryptonightR_instruction_mov31(void); +void CryptonightR_instruction_mov32(void); +void CryptonightR_instruction_mov33(void); +void CryptonightR_instruction_mov34(void); +void CryptonightR_instruction_mov35(void); +void CryptonightR_instruction_mov36(void); +void CryptonightR_instruction_mov37(void); +void CryptonightR_instruction_mov38(void); +void CryptonightR_instruction_mov39(void); +void CryptonightR_instruction_mov40(void); +void CryptonightR_instruction_mov41(void); +void CryptonightR_instruction_mov42(void); +void CryptonightR_instruction_mov43(void); +void CryptonightR_instruction_mov44(void); +void CryptonightR_instruction_mov45(void); +void CryptonightR_instruction_mov46(void); +void CryptonightR_instruction_mov47(void); +void CryptonightR_instruction_mov48(void); +void CryptonightR_instruction_mov49(void); +void CryptonightR_instruction_mov50(void); +void CryptonightR_instruction_mov51(void); +void CryptonightR_instruction_mov52(void); +void CryptonightR_instruction_mov53(void); +void CryptonightR_instruction_mov54(void); +void CryptonightR_instruction_mov55(void); +void CryptonightR_instruction_mov56(void); +void CryptonightR_instruction_mov57(void); +void CryptonightR_instruction_mov58(void); +void CryptonightR_instruction_mov59(void); +void CryptonightR_instruction_mov60(void); +void CryptonightR_instruction_mov61(void); +void CryptonightR_instruction_mov62(void); +void CryptonightR_instruction_mov63(void); +void CryptonightR_instruction_mov64(void); +void CryptonightR_instruction_mov65(void); +void CryptonightR_instruction_mov66(void); +void CryptonightR_instruction_mov67(void); +void CryptonightR_instruction_mov68(void); +void CryptonightR_instruction_mov69(void); +void CryptonightR_instruction_mov70(void); +void CryptonightR_instruction_mov71(void); +void CryptonightR_instruction_mov72(void); +void CryptonightR_instruction_mov73(void); +void CryptonightR_instruction_mov74(void); +void CryptonightR_instruction_mov75(void); +void CryptonightR_instruction_mov76(void); +void CryptonightR_instruction_mov77(void); +void CryptonightR_instruction_mov78(void); +void CryptonightR_instruction_mov79(void); +void CryptonightR_instruction_mov80(void); +void CryptonightR_instruction_mov81(void); +void CryptonightR_instruction_mov82(void); +void CryptonightR_instruction_mov83(void); +void CryptonightR_instruction_mov84(void); +void CryptonightR_instruction_mov85(void); +void CryptonightR_instruction_mov86(void); +void CryptonightR_instruction_mov87(void); +void CryptonightR_instruction_mov88(void); +void CryptonightR_instruction_mov89(void); +void CryptonightR_instruction_mov90(void); +void CryptonightR_instruction_mov91(void); +void CryptonightR_instruction_mov92(void); +void CryptonightR_instruction_mov93(void); +void CryptonightR_instruction_mov94(void); +void CryptonightR_instruction_mov95(void); +void CryptonightR_instruction_mov96(void); +void CryptonightR_instruction_mov97(void); +void CryptonightR_instruction_mov98(void); +void CryptonightR_instruction_mov99(void); +void CryptonightR_instruction_mov100(void); +void CryptonightR_instruction_mov101(void); +void CryptonightR_instruction_mov102(void); +void CryptonightR_instruction_mov103(void); +void CryptonightR_instruction_mov104(void); +void CryptonightR_instruction_mov105(void); +void CryptonightR_instruction_mov106(void); +void CryptonightR_instruction_mov107(void); +void CryptonightR_instruction_mov108(void); +void CryptonightR_instruction_mov109(void); +void CryptonightR_instruction_mov110(void); +void CryptonightR_instruction_mov111(void); +void CryptonightR_instruction_mov112(void); +void CryptonightR_instruction_mov113(void); +void CryptonightR_instruction_mov114(void); +void CryptonightR_instruction_mov115(void); +void CryptonightR_instruction_mov116(void); +void CryptonightR_instruction_mov117(void); +void CryptonightR_instruction_mov118(void); +void CryptonightR_instruction_mov119(void); +void CryptonightR_instruction_mov120(void); +void CryptonightR_instruction_mov121(void); +void CryptonightR_instruction_mov122(void); +void CryptonightR_instruction_mov123(void); +void CryptonightR_instruction_mov124(void); +void CryptonightR_instruction_mov125(void); +void CryptonightR_instruction_mov126(void); +void CryptonightR_instruction_mov127(void); +void CryptonightR_instruction_mov128(void); +void CryptonightR_instruction_mov129(void); +void CryptonightR_instruction_mov130(void); +void CryptonightR_instruction_mov131(void); +void CryptonightR_instruction_mov132(void); +void CryptonightR_instruction_mov133(void); +void CryptonightR_instruction_mov134(void); +void CryptonightR_instruction_mov135(void); +void CryptonightR_instruction_mov136(void); +void CryptonightR_instruction_mov137(void); +void CryptonightR_instruction_mov138(void); +void CryptonightR_instruction_mov139(void); +void CryptonightR_instruction_mov140(void); +void CryptonightR_instruction_mov141(void); +void CryptonightR_instruction_mov142(void); +void CryptonightR_instruction_mov143(void); +void CryptonightR_instruction_mov144(void); +void CryptonightR_instruction_mov145(void); +void CryptonightR_instruction_mov146(void); +void CryptonightR_instruction_mov147(void); +void CryptonightR_instruction_mov148(void); +void CryptonightR_instruction_mov149(void); +void CryptonightR_instruction_mov150(void); +void CryptonightR_instruction_mov151(void); +void CryptonightR_instruction_mov152(void); +void CryptonightR_instruction_mov153(void); +void CryptonightR_instruction_mov154(void); +void CryptonightR_instruction_mov155(void); +void CryptonightR_instruction_mov156(void); +void CryptonightR_instruction_mov157(void); +void CryptonightR_instruction_mov158(void); +void CryptonightR_instruction_mov159(void); +void CryptonightR_instruction_mov160(void); +void CryptonightR_instruction_mov161(void); +void CryptonightR_instruction_mov162(void); +void CryptonightR_instruction_mov163(void); +void CryptonightR_instruction_mov164(void); +void CryptonightR_instruction_mov165(void); +void CryptonightR_instruction_mov166(void); +void CryptonightR_instruction_mov167(void); +void CryptonightR_instruction_mov168(void); +void CryptonightR_instruction_mov169(void); +void CryptonightR_instruction_mov170(void); +void CryptonightR_instruction_mov171(void); +void CryptonightR_instruction_mov172(void); +void CryptonightR_instruction_mov173(void); +void CryptonightR_instruction_mov174(void); +void CryptonightR_instruction_mov175(void); +void CryptonightR_instruction_mov176(void); +void CryptonightR_instruction_mov177(void); +void CryptonightR_instruction_mov178(void); +void CryptonightR_instruction_mov179(void); +void CryptonightR_instruction_mov180(void); +void CryptonightR_instruction_mov181(void); +void CryptonightR_instruction_mov182(void); +void CryptonightR_instruction_mov183(void); +void CryptonightR_instruction_mov184(void); +void CryptonightR_instruction_mov185(void); +void CryptonightR_instruction_mov186(void); +void CryptonightR_instruction_mov187(void); +void CryptonightR_instruction_mov188(void); +void CryptonightR_instruction_mov189(void); +void CryptonightR_instruction_mov190(void); +void CryptonightR_instruction_mov191(void); +void CryptonightR_instruction_mov192(void); +void CryptonightR_instruction_mov193(void); +void CryptonightR_instruction_mov194(void); +void CryptonightR_instruction_mov195(void); +void CryptonightR_instruction_mov196(void); +void CryptonightR_instruction_mov197(void); +void CryptonightR_instruction_mov198(void); +void CryptonightR_instruction_mov199(void); +void CryptonightR_instruction_mov200(void); +void CryptonightR_instruction_mov201(void); +void CryptonightR_instruction_mov202(void); +void CryptonightR_instruction_mov203(void); +void CryptonightR_instruction_mov204(void); +void CryptonightR_instruction_mov205(void); +void CryptonightR_instruction_mov206(void); +void CryptonightR_instruction_mov207(void); +void CryptonightR_instruction_mov208(void); +void CryptonightR_instruction_mov209(void); +void CryptonightR_instruction_mov210(void); +void CryptonightR_instruction_mov211(void); +void CryptonightR_instruction_mov212(void); +void CryptonightR_instruction_mov213(void); +void CryptonightR_instruction_mov214(void); +void CryptonightR_instruction_mov215(void); +void CryptonightR_instruction_mov216(void); +void CryptonightR_instruction_mov217(void); +void CryptonightR_instruction_mov218(void); +void CryptonightR_instruction_mov219(void); +void CryptonightR_instruction_mov220(void); +void CryptonightR_instruction_mov221(void); +void CryptonightR_instruction_mov222(void); +void CryptonightR_instruction_mov223(void); +void CryptonightR_instruction_mov224(void); +void CryptonightR_instruction_mov225(void); +void CryptonightR_instruction_mov226(void); +void CryptonightR_instruction_mov227(void); +void CryptonightR_instruction_mov228(void); +void CryptonightR_instruction_mov229(void); +void CryptonightR_instruction_mov230(void); +void CryptonightR_instruction_mov231(void); +void CryptonightR_instruction_mov232(void); +void CryptonightR_instruction_mov233(void); +void CryptonightR_instruction_mov234(void); +void CryptonightR_instruction_mov235(void); +void CryptonightR_instruction_mov236(void); +void CryptonightR_instruction_mov237(void); +void CryptonightR_instruction_mov238(void); +void CryptonightR_instruction_mov239(void); +void CryptonightR_instruction_mov240(void); +void CryptonightR_instruction_mov241(void); +void CryptonightR_instruction_mov242(void); +void CryptonightR_instruction_mov243(void); +void CryptonightR_instruction_mov244(void); +void CryptonightR_instruction_mov245(void); +void CryptonightR_instruction_mov246(void); +void CryptonightR_instruction_mov247(void); +void CryptonightR_instruction_mov248(void); +void CryptonightR_instruction_mov249(void); +void CryptonightR_instruction_mov250(void); +void CryptonightR_instruction_mov251(void); +void CryptonightR_instruction_mov252(void); +void CryptonightR_instruction_mov253(void); +void CryptonightR_instruction_mov254(void); +void CryptonightR_instruction_mov255(void); +void CryptonightR_instruction_mov256(void); + +const void* instructions[257] = { + CryptonightR_instruction0, + CryptonightR_instruction1, + CryptonightR_instruction2, + CryptonightR_instruction3, + CryptonightR_instruction4, + CryptonightR_instruction5, + CryptonightR_instruction6, + CryptonightR_instruction7, + CryptonightR_instruction8, + CryptonightR_instruction9, + CryptonightR_instruction10, + CryptonightR_instruction11, + CryptonightR_instruction12, + CryptonightR_instruction13, + CryptonightR_instruction14, + CryptonightR_instruction15, + CryptonightR_instruction16, + CryptonightR_instruction17, + CryptonightR_instruction18, + CryptonightR_instruction19, + CryptonightR_instruction20, + CryptonightR_instruction21, + CryptonightR_instruction22, + CryptonightR_instruction23, + CryptonightR_instruction24, + CryptonightR_instruction25, + CryptonightR_instruction26, + CryptonightR_instruction27, + CryptonightR_instruction28, + CryptonightR_instruction29, + CryptonightR_instruction30, + CryptonightR_instruction31, + CryptonightR_instruction32, + CryptonightR_instruction33, + CryptonightR_instruction34, + CryptonightR_instruction35, + CryptonightR_instruction36, + CryptonightR_instruction37, + CryptonightR_instruction38, + CryptonightR_instruction39, + CryptonightR_instruction40, + CryptonightR_instruction41, + CryptonightR_instruction42, + CryptonightR_instruction43, + CryptonightR_instruction44, + CryptonightR_instruction45, + CryptonightR_instruction46, + CryptonightR_instruction47, + CryptonightR_instruction48, + CryptonightR_instruction49, + CryptonightR_instruction50, + CryptonightR_instruction51, + CryptonightR_instruction52, + CryptonightR_instruction53, + CryptonightR_instruction54, + CryptonightR_instruction55, + CryptonightR_instruction56, + CryptonightR_instruction57, + CryptonightR_instruction58, + CryptonightR_instruction59, + CryptonightR_instruction60, + CryptonightR_instruction61, + CryptonightR_instruction62, + CryptonightR_instruction63, + CryptonightR_instruction64, + CryptonightR_instruction65, + CryptonightR_instruction66, + CryptonightR_instruction67, + CryptonightR_instruction68, + CryptonightR_instruction69, + CryptonightR_instruction70, + CryptonightR_instruction71, + CryptonightR_instruction72, + CryptonightR_instruction73, + CryptonightR_instruction74, + CryptonightR_instruction75, + CryptonightR_instruction76, + CryptonightR_instruction77, + CryptonightR_instruction78, + CryptonightR_instruction79, + CryptonightR_instruction80, + CryptonightR_instruction81, + CryptonightR_instruction82, + CryptonightR_instruction83, + CryptonightR_instruction84, + CryptonightR_instruction85, + CryptonightR_instruction86, + CryptonightR_instruction87, + CryptonightR_instruction88, + CryptonightR_instruction89, + CryptonightR_instruction90, + CryptonightR_instruction91, + CryptonightR_instruction92, + CryptonightR_instruction93, + CryptonightR_instruction94, + CryptonightR_instruction95, + CryptonightR_instruction96, + CryptonightR_instruction97, + CryptonightR_instruction98, + CryptonightR_instruction99, + CryptonightR_instruction100, + CryptonightR_instruction101, + CryptonightR_instruction102, + CryptonightR_instruction103, + CryptonightR_instruction104, + CryptonightR_instruction105, + CryptonightR_instruction106, + CryptonightR_instruction107, + CryptonightR_instruction108, + CryptonightR_instruction109, + CryptonightR_instruction110, + CryptonightR_instruction111, + CryptonightR_instruction112, + CryptonightR_instruction113, + CryptonightR_instruction114, + CryptonightR_instruction115, + CryptonightR_instruction116, + CryptonightR_instruction117, + CryptonightR_instruction118, + CryptonightR_instruction119, + CryptonightR_instruction120, + CryptonightR_instruction121, + CryptonightR_instruction122, + CryptonightR_instruction123, + CryptonightR_instruction124, + CryptonightR_instruction125, + CryptonightR_instruction126, + CryptonightR_instruction127, + CryptonightR_instruction128, + CryptonightR_instruction129, + CryptonightR_instruction130, + CryptonightR_instruction131, + CryptonightR_instruction132, + CryptonightR_instruction133, + CryptonightR_instruction134, + CryptonightR_instruction135, + CryptonightR_instruction136, + CryptonightR_instruction137, + CryptonightR_instruction138, + CryptonightR_instruction139, + CryptonightR_instruction140, + CryptonightR_instruction141, + CryptonightR_instruction142, + CryptonightR_instruction143, + CryptonightR_instruction144, + CryptonightR_instruction145, + CryptonightR_instruction146, + CryptonightR_instruction147, + CryptonightR_instruction148, + CryptonightR_instruction149, + CryptonightR_instruction150, + CryptonightR_instruction151, + CryptonightR_instruction152, + CryptonightR_instruction153, + CryptonightR_instruction154, + CryptonightR_instruction155, + CryptonightR_instruction156, + CryptonightR_instruction157, + CryptonightR_instruction158, + CryptonightR_instruction159, + CryptonightR_instruction160, + CryptonightR_instruction161, + CryptonightR_instruction162, + CryptonightR_instruction163, + CryptonightR_instruction164, + CryptonightR_instruction165, + CryptonightR_instruction166, + CryptonightR_instruction167, + CryptonightR_instruction168, + CryptonightR_instruction169, + CryptonightR_instruction170, + CryptonightR_instruction171, + CryptonightR_instruction172, + CryptonightR_instruction173, + CryptonightR_instruction174, + CryptonightR_instruction175, + CryptonightR_instruction176, + CryptonightR_instruction177, + CryptonightR_instruction178, + CryptonightR_instruction179, + CryptonightR_instruction180, + CryptonightR_instruction181, + CryptonightR_instruction182, + CryptonightR_instruction183, + CryptonightR_instruction184, + CryptonightR_instruction185, + CryptonightR_instruction186, + CryptonightR_instruction187, + CryptonightR_instruction188, + CryptonightR_instruction189, + CryptonightR_instruction190, + CryptonightR_instruction191, + CryptonightR_instruction192, + CryptonightR_instruction193, + CryptonightR_instruction194, + CryptonightR_instruction195, + CryptonightR_instruction196, + CryptonightR_instruction197, + CryptonightR_instruction198, + CryptonightR_instruction199, + CryptonightR_instruction200, + CryptonightR_instruction201, + CryptonightR_instruction202, + CryptonightR_instruction203, + CryptonightR_instruction204, + CryptonightR_instruction205, + CryptonightR_instruction206, + CryptonightR_instruction207, + CryptonightR_instruction208, + CryptonightR_instruction209, + CryptonightR_instruction210, + CryptonightR_instruction211, + CryptonightR_instruction212, + CryptonightR_instruction213, + CryptonightR_instruction214, + CryptonightR_instruction215, + CryptonightR_instruction216, + CryptonightR_instruction217, + CryptonightR_instruction218, + CryptonightR_instruction219, + CryptonightR_instruction220, + CryptonightR_instruction221, + CryptonightR_instruction222, + CryptonightR_instruction223, + CryptonightR_instruction224, + CryptonightR_instruction225, + CryptonightR_instruction226, + CryptonightR_instruction227, + CryptonightR_instruction228, + CryptonightR_instruction229, + CryptonightR_instruction230, + CryptonightR_instruction231, + CryptonightR_instruction232, + CryptonightR_instruction233, + CryptonightR_instruction234, + CryptonightR_instruction235, + CryptonightR_instruction236, + CryptonightR_instruction237, + CryptonightR_instruction238, + CryptonightR_instruction239, + CryptonightR_instruction240, + CryptonightR_instruction241, + CryptonightR_instruction242, + CryptonightR_instruction243, + CryptonightR_instruction244, + CryptonightR_instruction245, + CryptonightR_instruction246, + CryptonightR_instruction247, + CryptonightR_instruction248, + CryptonightR_instruction249, + CryptonightR_instruction250, + CryptonightR_instruction251, + CryptonightR_instruction252, + CryptonightR_instruction253, + CryptonightR_instruction254, + CryptonightR_instruction255, + CryptonightR_instruction256, +}; + +const void* instructions_mov[257] = { + CryptonightR_instruction_mov0, + CryptonightR_instruction_mov1, + CryptonightR_instruction_mov2, + CryptonightR_instruction_mov3, + CryptonightR_instruction_mov4, + CryptonightR_instruction_mov5, + CryptonightR_instruction_mov6, + CryptonightR_instruction_mov7, + CryptonightR_instruction_mov8, + CryptonightR_instruction_mov9, + CryptonightR_instruction_mov10, + CryptonightR_instruction_mov11, + CryptonightR_instruction_mov12, + CryptonightR_instruction_mov13, + CryptonightR_instruction_mov14, + CryptonightR_instruction_mov15, + CryptonightR_instruction_mov16, + CryptonightR_instruction_mov17, + CryptonightR_instruction_mov18, + CryptonightR_instruction_mov19, + CryptonightR_instruction_mov20, + CryptonightR_instruction_mov21, + CryptonightR_instruction_mov22, + CryptonightR_instruction_mov23, + CryptonightR_instruction_mov24, + CryptonightR_instruction_mov25, + CryptonightR_instruction_mov26, + CryptonightR_instruction_mov27, + CryptonightR_instruction_mov28, + CryptonightR_instruction_mov29, + CryptonightR_instruction_mov30, + CryptonightR_instruction_mov31, + CryptonightR_instruction_mov32, + CryptonightR_instruction_mov33, + CryptonightR_instruction_mov34, + CryptonightR_instruction_mov35, + CryptonightR_instruction_mov36, + CryptonightR_instruction_mov37, + CryptonightR_instruction_mov38, + CryptonightR_instruction_mov39, + CryptonightR_instruction_mov40, + CryptonightR_instruction_mov41, + CryptonightR_instruction_mov42, + CryptonightR_instruction_mov43, + CryptonightR_instruction_mov44, + CryptonightR_instruction_mov45, + CryptonightR_instruction_mov46, + CryptonightR_instruction_mov47, + CryptonightR_instruction_mov48, + CryptonightR_instruction_mov49, + CryptonightR_instruction_mov50, + CryptonightR_instruction_mov51, + CryptonightR_instruction_mov52, + CryptonightR_instruction_mov53, + CryptonightR_instruction_mov54, + CryptonightR_instruction_mov55, + CryptonightR_instruction_mov56, + CryptonightR_instruction_mov57, + CryptonightR_instruction_mov58, + CryptonightR_instruction_mov59, + CryptonightR_instruction_mov60, + CryptonightR_instruction_mov61, + CryptonightR_instruction_mov62, + CryptonightR_instruction_mov63, + CryptonightR_instruction_mov64, + CryptonightR_instruction_mov65, + CryptonightR_instruction_mov66, + CryptonightR_instruction_mov67, + CryptonightR_instruction_mov68, + CryptonightR_instruction_mov69, + CryptonightR_instruction_mov70, + CryptonightR_instruction_mov71, + CryptonightR_instruction_mov72, + CryptonightR_instruction_mov73, + CryptonightR_instruction_mov74, + CryptonightR_instruction_mov75, + CryptonightR_instruction_mov76, + CryptonightR_instruction_mov77, + CryptonightR_instruction_mov78, + CryptonightR_instruction_mov79, + CryptonightR_instruction_mov80, + CryptonightR_instruction_mov81, + CryptonightR_instruction_mov82, + CryptonightR_instruction_mov83, + CryptonightR_instruction_mov84, + CryptonightR_instruction_mov85, + CryptonightR_instruction_mov86, + CryptonightR_instruction_mov87, + CryptonightR_instruction_mov88, + CryptonightR_instruction_mov89, + CryptonightR_instruction_mov90, + CryptonightR_instruction_mov91, + CryptonightR_instruction_mov92, + CryptonightR_instruction_mov93, + CryptonightR_instruction_mov94, + CryptonightR_instruction_mov95, + CryptonightR_instruction_mov96, + CryptonightR_instruction_mov97, + CryptonightR_instruction_mov98, + CryptonightR_instruction_mov99, + CryptonightR_instruction_mov100, + CryptonightR_instruction_mov101, + CryptonightR_instruction_mov102, + CryptonightR_instruction_mov103, + CryptonightR_instruction_mov104, + CryptonightR_instruction_mov105, + CryptonightR_instruction_mov106, + CryptonightR_instruction_mov107, + CryptonightR_instruction_mov108, + CryptonightR_instruction_mov109, + CryptonightR_instruction_mov110, + CryptonightR_instruction_mov111, + CryptonightR_instruction_mov112, + CryptonightR_instruction_mov113, + CryptonightR_instruction_mov114, + CryptonightR_instruction_mov115, + CryptonightR_instruction_mov116, + CryptonightR_instruction_mov117, + CryptonightR_instruction_mov118, + CryptonightR_instruction_mov119, + CryptonightR_instruction_mov120, + CryptonightR_instruction_mov121, + CryptonightR_instruction_mov122, + CryptonightR_instruction_mov123, + CryptonightR_instruction_mov124, + CryptonightR_instruction_mov125, + CryptonightR_instruction_mov126, + CryptonightR_instruction_mov127, + CryptonightR_instruction_mov128, + CryptonightR_instruction_mov129, + CryptonightR_instruction_mov130, + CryptonightR_instruction_mov131, + CryptonightR_instruction_mov132, + CryptonightR_instruction_mov133, + CryptonightR_instruction_mov134, + CryptonightR_instruction_mov135, + CryptonightR_instruction_mov136, + CryptonightR_instruction_mov137, + CryptonightR_instruction_mov138, + CryptonightR_instruction_mov139, + CryptonightR_instruction_mov140, + CryptonightR_instruction_mov141, + CryptonightR_instruction_mov142, + CryptonightR_instruction_mov143, + CryptonightR_instruction_mov144, + CryptonightR_instruction_mov145, + CryptonightR_instruction_mov146, + CryptonightR_instruction_mov147, + CryptonightR_instruction_mov148, + CryptonightR_instruction_mov149, + CryptonightR_instruction_mov150, + CryptonightR_instruction_mov151, + CryptonightR_instruction_mov152, + CryptonightR_instruction_mov153, + CryptonightR_instruction_mov154, + CryptonightR_instruction_mov155, + CryptonightR_instruction_mov156, + CryptonightR_instruction_mov157, + CryptonightR_instruction_mov158, + CryptonightR_instruction_mov159, + CryptonightR_instruction_mov160, + CryptonightR_instruction_mov161, + CryptonightR_instruction_mov162, + CryptonightR_instruction_mov163, + CryptonightR_instruction_mov164, + CryptonightR_instruction_mov165, + CryptonightR_instruction_mov166, + CryptonightR_instruction_mov167, + CryptonightR_instruction_mov168, + CryptonightR_instruction_mov169, + CryptonightR_instruction_mov170, + CryptonightR_instruction_mov171, + CryptonightR_instruction_mov172, + CryptonightR_instruction_mov173, + CryptonightR_instruction_mov174, + CryptonightR_instruction_mov175, + CryptonightR_instruction_mov176, + CryptonightR_instruction_mov177, + CryptonightR_instruction_mov178, + CryptonightR_instruction_mov179, + CryptonightR_instruction_mov180, + CryptonightR_instruction_mov181, + CryptonightR_instruction_mov182, + CryptonightR_instruction_mov183, + CryptonightR_instruction_mov184, + CryptonightR_instruction_mov185, + CryptonightR_instruction_mov186, + CryptonightR_instruction_mov187, + CryptonightR_instruction_mov188, + CryptonightR_instruction_mov189, + CryptonightR_instruction_mov190, + CryptonightR_instruction_mov191, + CryptonightR_instruction_mov192, + CryptonightR_instruction_mov193, + CryptonightR_instruction_mov194, + CryptonightR_instruction_mov195, + CryptonightR_instruction_mov196, + CryptonightR_instruction_mov197, + CryptonightR_instruction_mov198, + CryptonightR_instruction_mov199, + CryptonightR_instruction_mov200, + CryptonightR_instruction_mov201, + CryptonightR_instruction_mov202, + CryptonightR_instruction_mov203, + CryptonightR_instruction_mov204, + CryptonightR_instruction_mov205, + CryptonightR_instruction_mov206, + CryptonightR_instruction_mov207, + CryptonightR_instruction_mov208, + CryptonightR_instruction_mov209, + CryptonightR_instruction_mov210, + CryptonightR_instruction_mov211, + CryptonightR_instruction_mov212, + CryptonightR_instruction_mov213, + CryptonightR_instruction_mov214, + CryptonightR_instruction_mov215, + CryptonightR_instruction_mov216, + CryptonightR_instruction_mov217, + CryptonightR_instruction_mov218, + CryptonightR_instruction_mov219, + CryptonightR_instruction_mov220, + CryptonightR_instruction_mov221, + CryptonightR_instruction_mov222, + CryptonightR_instruction_mov223, + CryptonightR_instruction_mov224, + CryptonightR_instruction_mov225, + CryptonightR_instruction_mov226, + CryptonightR_instruction_mov227, + CryptonightR_instruction_mov228, + CryptonightR_instruction_mov229, + CryptonightR_instruction_mov230, + CryptonightR_instruction_mov231, + CryptonightR_instruction_mov232, + CryptonightR_instruction_mov233, + CryptonightR_instruction_mov234, + CryptonightR_instruction_mov235, + CryptonightR_instruction_mov236, + CryptonightR_instruction_mov237, + CryptonightR_instruction_mov238, + CryptonightR_instruction_mov239, + CryptonightR_instruction_mov240, + CryptonightR_instruction_mov241, + CryptonightR_instruction_mov242, + CryptonightR_instruction_mov243, + CryptonightR_instruction_mov244, + CryptonightR_instruction_mov245, + CryptonightR_instruction_mov246, + CryptonightR_instruction_mov247, + CryptonightR_instruction_mov248, + CryptonightR_instruction_mov249, + CryptonightR_instruction_mov250, + CryptonightR_instruction_mov251, + CryptonightR_instruction_mov252, + CryptonightR_instruction_mov253, + CryptonightR_instruction_mov254, + CryptonightR_instruction_mov255, + CryptonightR_instruction_mov256, +}; + +#endif // CRYPTONIGHTR_TEMPLATE_H diff --git a/src/crypto/chacha.h b/src/crypto/chacha.h index 6e85ad0e9..0610f7051 100644 --- a/src/crypto/chacha.h +++ b/src/crypto/chacha.h @@ -73,18 +73,18 @@ namespace crypto { inline void generate_chacha_key(const void *data, size_t size, chacha_key& key, uint64_t kdf_rounds) { static_assert(sizeof(chacha_key) <= sizeof(hash), "Size of hash must be at least that of chacha_key"); epee::mlocked<tools::scrubbed_arr<char, HASH_SIZE>> pwd_hash; - crypto::cn_slow_hash(data, size, pwd_hash.data(), 0/*variant*/, 0/*prehashed*/); + crypto::cn_slow_hash(data, size, pwd_hash.data(), 0/*variant*/, 0/*prehashed*/, 0/*height*/); for (uint64_t n = 1; n < kdf_rounds; ++n) - crypto::cn_slow_hash(pwd_hash.data(), pwd_hash.size(), pwd_hash.data(), 0/*variant*/, 0/*prehashed*/); + crypto::cn_slow_hash(pwd_hash.data(), pwd_hash.size(), pwd_hash.data(), 0/*variant*/, 0/*prehashed*/, 0/*height*/); memcpy(&unwrap(unwrap(key)), pwd_hash.data(), sizeof(key)); } inline void generate_chacha_key_prehashed(const void *data, size_t size, chacha_key& key, uint64_t kdf_rounds) { static_assert(sizeof(chacha_key) <= sizeof(hash), "Size of hash must be at least that of chacha_key"); epee::mlocked<tools::scrubbed_arr<char, HASH_SIZE>> pwd_hash; - crypto::cn_slow_hash(data, size, pwd_hash.data(), 0/*variant*/, 1/*prehashed*/); + crypto::cn_slow_hash(data, size, pwd_hash.data(), 0/*variant*/, 1/*prehashed*/, 0/*height*/); for (uint64_t n = 1; n < kdf_rounds; ++n) - crypto::cn_slow_hash(pwd_hash.data(), pwd_hash.size(), pwd_hash.data(), 0/*variant*/, 0/*prehashed*/); + crypto::cn_slow_hash(pwd_hash.data(), pwd_hash.size(), pwd_hash.data(), 0/*variant*/, 0/*prehashed*/, 0/*height*/); memcpy(&unwrap(unwrap(key)), pwd_hash.data(), sizeof(key)); } diff --git a/src/crypto/hash-ops.h b/src/crypto/hash-ops.h index 77b52e2d4..ba7ece0f5 100644 --- a/src/crypto/hash-ops.h +++ b/src/crypto/hash-ops.h @@ -79,7 +79,7 @@ enum { }; void cn_fast_hash(const void *data, size_t length, char *hash); -void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed); +void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed, uint64_t height); void hash_extra_blake(const void *data, size_t length, char *hash); void hash_extra_groestl(const void *data, size_t length, char *hash); diff --git a/src/crypto/hash.h b/src/crypto/hash.h index 995e2294e..165fe6bb0 100644 --- a/src/crypto/hash.h +++ b/src/crypto/hash.h @@ -71,12 +71,12 @@ namespace crypto { return h; } - inline void cn_slow_hash(const void *data, std::size_t length, hash &hash, int variant = 0) { - cn_slow_hash(data, length, reinterpret_cast<char *>(&hash), variant, 0/*prehashed*/); + inline void cn_slow_hash(const void *data, std::size_t length, hash &hash, int variant = 0, uint64_t height = 0) { + cn_slow_hash(data, length, reinterpret_cast<char *>(&hash), variant, 0/*prehashed*/, height); } - inline void cn_slow_hash_prehashed(const void *data, std::size_t length, hash &hash, int variant = 0) { - cn_slow_hash(data, length, reinterpret_cast<char *>(&hash), variant, 1/*prehashed*/); + inline void cn_slow_hash_prehashed(const void *data, std::size_t length, hash &hash, int variant = 0, uint64_t height = 0) { + cn_slow_hash(data, length, reinterpret_cast<char *>(&hash), variant, 1/*prehashed*/, height); } inline void tree_hash(const hash *hashes, std::size_t count, hash &root_hash) { diff --git a/src/crypto/slow-hash.c b/src/crypto/slow-hash.c index ae0bd4e98..2a8ddb59c 100644 --- a/src/crypto/slow-hash.c +++ b/src/crypto/slow-hash.c @@ -39,6 +39,11 @@ #include "hash-ops.h" #include "oaes_lib.h" #include "variant2_int_sqrt.h" +#include "variant4_random_math.h" +#include "CryptonightR_JIT.h" + +#include <errno.h> +#include <string.h> #define MEMORY (1 << 21) // 2MB scratchpad #define ITER (1 << 20) @@ -50,6 +55,16 @@ extern void aesb_single_round(const uint8_t *in, uint8_t *out, const uint8_t *expandedKey); extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *expandedKey); +static void local_abort(const char *msg) +{ + fprintf(stderr, "%s\n", msg); +#ifdef NDEBUG + _exit(1); +#else + abort(); +#endif +} + #define VARIANT1_1(p) \ do if (variant == 1) \ { \ @@ -116,48 +131,74 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex #define VARIANT2_SHUFFLE_ADD_SSE2(base_ptr, offset) \ do if (variant >= 2) \ { \ - const __m128i chunk1 = _mm_load_si128((__m128i *)((base_ptr) + ((offset) ^ 0x10))); \ + __m128i chunk1 = _mm_load_si128((__m128i *)((base_ptr) + ((offset) ^ 0x10))); \ const __m128i chunk2 = _mm_load_si128((__m128i *)((base_ptr) + ((offset) ^ 0x20))); \ const __m128i chunk3 = _mm_load_si128((__m128i *)((base_ptr) + ((offset) ^ 0x30))); \ _mm_store_si128((__m128i *)((base_ptr) + ((offset) ^ 0x10)), _mm_add_epi64(chunk3, _b1)); \ _mm_store_si128((__m128i *)((base_ptr) + ((offset) ^ 0x20)), _mm_add_epi64(chunk1, _b)); \ _mm_store_si128((__m128i *)((base_ptr) + ((offset) ^ 0x30)), _mm_add_epi64(chunk2, _a)); \ + if (variant >= 4) \ + { \ + chunk1 = _mm_xor_si128(chunk1, chunk2); \ + _c = _mm_xor_si128(_c, chunk3); \ + _c = _mm_xor_si128(_c, chunk1); \ + } \ } while (0) #define VARIANT2_SHUFFLE_ADD_NEON(base_ptr, offset) \ do if (variant >= 2) \ { \ - const uint64x2_t chunk1 = vld1q_u64(U64((base_ptr) + ((offset) ^ 0x10))); \ + uint64x2_t chunk1 = vld1q_u64(U64((base_ptr) + ((offset) ^ 0x10))); \ const uint64x2_t chunk2 = vld1q_u64(U64((base_ptr) + ((offset) ^ 0x20))); \ const uint64x2_t chunk3 = vld1q_u64(U64((base_ptr) + ((offset) ^ 0x30))); \ vst1q_u64(U64((base_ptr) + ((offset) ^ 0x10)), vaddq_u64(chunk3, vreinterpretq_u64_u8(_b1))); \ vst1q_u64(U64((base_ptr) + ((offset) ^ 0x20)), vaddq_u64(chunk1, vreinterpretq_u64_u8(_b))); \ vst1q_u64(U64((base_ptr) + ((offset) ^ 0x30)), vaddq_u64(chunk2, vreinterpretq_u64_u8(_a))); \ + if (variant >= 4) \ + { \ + chunk1 = veorq_u64(chunk1, chunk2); \ + _c = vreinterpretq_u8_u64(veorq_u64(vreinterpretq_u64_u8(_c), chunk3)); \ + _c = vreinterpretq_u8_u64(veorq_u64(vreinterpretq_u64_u8(_c), chunk1)); \ + } \ } while (0) -#define VARIANT2_PORTABLE_SHUFFLE_ADD(base_ptr, offset) \ +#define VARIANT2_PORTABLE_SHUFFLE_ADD(out, a_, base_ptr, offset) \ do if (variant >= 2) \ { \ uint64_t* chunk1 = U64((base_ptr) + ((offset) ^ 0x10)); \ uint64_t* chunk2 = U64((base_ptr) + ((offset) ^ 0x20)); \ uint64_t* chunk3 = U64((base_ptr) + ((offset) ^ 0x30)); \ \ - const uint64_t chunk1_old[2] = { chunk1[0], chunk1[1] }; \ + uint64_t chunk1_old[2] = { SWAP64LE(chunk1[0]), SWAP64LE(chunk1[1]) }; \ + const uint64_t chunk2_old[2] = { SWAP64LE(chunk2[0]), SWAP64LE(chunk2[1]) }; \ + const uint64_t chunk3_old[2] = { SWAP64LE(chunk3[0]), SWAP64LE(chunk3[1]) }; \ \ uint64_t b1[2]; \ memcpy_swap64le(b1, b + 16, 2); \ - chunk1[0] = SWAP64LE(SWAP64LE(chunk3[0]) + b1[0]); \ - chunk1[1] = SWAP64LE(SWAP64LE(chunk3[1]) + b1[1]); \ + chunk1[0] = SWAP64LE(chunk3_old[0] + b1[0]); \ + chunk1[1] = SWAP64LE(chunk3_old[1] + b1[1]); \ \ uint64_t a0[2]; \ - memcpy_swap64le(a0, a, 2); \ - chunk3[0] = SWAP64LE(SWAP64LE(chunk2[0]) + a0[0]); \ - chunk3[1] = SWAP64LE(SWAP64LE(chunk2[1]) + a0[1]); \ + memcpy_swap64le(a0, a_, 2); \ + chunk3[0] = SWAP64LE(chunk2_old[0] + a0[0]); \ + chunk3[1] = SWAP64LE(chunk2_old[1] + a0[1]); \ \ uint64_t b0[2]; \ memcpy_swap64le(b0, b, 2); \ - chunk2[0] = SWAP64LE(SWAP64LE(chunk1_old[0]) + b0[0]); \ + chunk2[0] = SWAP64LE(chunk1_old[0] + b0[0]); \ chunk2[1] = SWAP64LE(SWAP64LE(chunk1_old[1]) + b0[1]); \ + if (variant >= 4) \ + { \ + uint64_t out_copy[2]; \ + memcpy_swap64le(out_copy, out, 2); \ + chunk1_old[0] ^= chunk2_old[0]; \ + chunk1_old[1] ^= chunk2_old[1]; \ + out_copy[0] ^= chunk3_old[0]; \ + out_copy[1] ^= chunk3_old[1]; \ + out_copy[0] ^= chunk1_old[0]; \ + out_copy[1] ^= chunk1_old[1]; \ + memcpy_swap64le(out, out_copy, 2); \ + } \ } while (0) #define VARIANT2_INTEGER_MATH_DIVISION_STEP(b, ptr) \ @@ -172,7 +213,7 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex const uint64_t sqrt_input = SWAP64LE(((uint64_t*)(ptr))[0]) + division_result #define VARIANT2_INTEGER_MATH_SSE2(b, ptr) \ - do if (variant >= 2) \ + do if ((variant == 2) || (variant == 3)) \ { \ VARIANT2_INTEGER_MATH_DIVISION_STEP(b, ptr); \ VARIANT2_INTEGER_MATH_SQRT_STEP_SSE2(); \ @@ -182,7 +223,7 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex #if defined DBL_MANT_DIG && (DBL_MANT_DIG >= 50) // double precision floating point type has enough bits of precision on current platform #define VARIANT2_PORTABLE_INTEGER_MATH(b, ptr) \ - do if (variant >= 2) \ + do if ((variant == 2) || (variant == 3)) \ { \ VARIANT2_INTEGER_MATH_DIVISION_STEP(b, ptr); \ VARIANT2_INTEGER_MATH_SQRT_STEP_FP64(); \ @@ -192,7 +233,7 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex // double precision floating point type is not good enough on current platform // fall back to the reference code (integer only) #define VARIANT2_PORTABLE_INTEGER_MATH(b, ptr) \ - do if (variant >= 2) \ + do if ((variant == 2) || (variant == 3)) \ { \ VARIANT2_INTEGER_MATH_DIVISION_STEP(b, ptr); \ VARIANT2_INTEGER_MATH_SQRT_STEP_REF(); \ @@ -200,13 +241,13 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex #endif #define VARIANT2_2_PORTABLE() \ - if (variant >= 2) { \ + if (variant == 2 || variant == 3) { \ xor_blocks(long_state + (j ^ 0x10), d); \ xor_blocks(d, long_state + (j ^ 0x20)); \ } #define VARIANT2_2() \ - do if (variant >= 2) \ + do if (variant == 2 || variant == 3) \ { \ *U64(hp_state + (j ^ 0x10)) ^= SWAP64LE(hi); \ *(U64(hp_state + (j ^ 0x10)) + 1) ^= SWAP64LE(lo); \ @@ -214,6 +255,68 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex lo ^= SWAP64LE(*(U64(hp_state + (j ^ 0x20)) + 1)); \ } while (0) +#define V4_REG_LOAD(dst, src) \ + do { \ + memcpy((dst), (src), sizeof(v4_reg)); \ + if (sizeof(v4_reg) == sizeof(uint32_t)) \ + *(dst) = SWAP32LE(*(dst)); \ + else \ + *(dst) = SWAP64LE(*(dst)); \ + } while (0) + +#define VARIANT4_RANDOM_MATH_INIT() \ + v4_reg r[9]; \ + struct V4_Instruction code[NUM_INSTRUCTIONS_MAX + 1]; \ + int jit = use_v4_jit(); \ + do if (variant >= 4) \ + { \ + for (int i = 0; i < 4; ++i) \ + V4_REG_LOAD(r + i, (uint8_t*)(state.hs.w + 12) + sizeof(v4_reg) * i); \ + v4_random_math_init(code, height); \ + if (jit) \ + { \ + int ret = v4_generate_JIT_code(code, hp_jitfunc, 4096); \ + if (ret < 0) \ + local_abort("Error generating CryptonightR code"); \ + } \ + } while (0) + +#define VARIANT4_RANDOM_MATH(a, b, r, _b, _b1) \ + do if (variant >= 4) \ + { \ + uint64_t t[2]; \ + memcpy(t, b, sizeof(uint64_t)); \ + \ + if (sizeof(v4_reg) == sizeof(uint32_t)) \ + t[0] ^= SWAP64LE((r[0] + r[1]) | ((uint64_t)(r[2] + r[3]) << 32)); \ + else \ + t[0] ^= SWAP64LE((r[0] + r[1]) ^ (r[2] + r[3])); \ + \ + memcpy(b, t, sizeof(uint64_t)); \ + \ + V4_REG_LOAD(r + 4, a); \ + V4_REG_LOAD(r + 5, (uint64_t*)(a) + 1); \ + V4_REG_LOAD(r + 6, _b); \ + V4_REG_LOAD(r + 7, _b1); \ + V4_REG_LOAD(r + 8, (uint64_t*)(_b1) + 1); \ + \ + if (jit) \ + (*hp_jitfunc)(r); \ + else \ + v4_random_math(code, r); \ + \ + memcpy(t, a, sizeof(uint64_t) * 2); \ + \ + if (sizeof(v4_reg) == sizeof(uint32_t)) { \ + t[0] ^= SWAP64LE(r[2] | ((uint64_t)(r[3]) << 32)); \ + t[1] ^= SWAP64LE(r[0] | ((uint64_t)(r[1]) << 32)); \ + } else { \ + t[0] ^= SWAP64LE(r[2] ^ r[3]); \ + t[1] ^= SWAP64LE(r[0] ^ r[1]); \ + } \ + memcpy(a, t, sizeof(uint64_t) * 2); \ + } while (0) + #if !defined NO_AES && (defined(__x86_64__) || (defined(_MSC_VER) && defined(_WIN64))) // Optimised code below, uses x86-specific intrinsics, SSE2, AES-NI @@ -298,6 +401,7 @@ extern void aesb_pseudo_round(const uint8_t *in, uint8_t *out, const uint8_t *ex p = U64(&hp_state[j]); \ b[0] = p[0]; b[1] = p[1]; \ VARIANT2_INTEGER_MATH_SSE2(b, c); \ + VARIANT4_RANDOM_MATH(a, b, r, &_b, &_b1); \ __mul(); \ VARIANT2_2(); \ VARIANT2_SHUFFLE_ADD_SSE2(hp_state, j); \ @@ -329,6 +433,9 @@ union cn_slow_hash_state THREADV uint8_t *hp_state = NULL; THREADV int hp_allocated = 0; +THREADV v4_random_math_JIT_func hp_jitfunc = NULL; +THREADV uint8_t *hp_jitfunc_memory = NULL; +THREADV int hp_jitfunc_allocated = 0; #if defined(_MSC_VER) #define cpuid(info,x) __cpuidex(info,x,0) @@ -387,6 +494,31 @@ STATIC INLINE int force_software_aes(void) return use; } +volatile int use_v4_jit_flag = -1; + +STATIC INLINE int use_v4_jit(void) +{ +#if defined(__x86_64__) + + if (use_v4_jit_flag != -1) + return use_v4_jit_flag; + + const char *env = getenv("MONERO_USE_CNV4_JIT"); + if (!env) { + use_v4_jit_flag = 0; + } + else if (!strcmp(env, "0") || !strcmp(env, "no")) { + use_v4_jit_flag = 0; + } + else { + use_v4_jit_flag = 1; + } + return use_v4_jit_flag; +#else + return 0; +#endif +} + STATIC INLINE int check_aes_hw(void) { int cpuid_results[4]; @@ -638,6 +770,33 @@ void slow_hash_allocate_state(void) hp_allocated = 0; hp_state = (uint8_t *) malloc(MEMORY); } + + +#if defined(_MSC_VER) || defined(__MINGW32__) + hp_jitfunc_memory = (uint8_t *) VirtualAlloc(hp_jitfunc_memory, 4096 + 4095, + MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE); +#else +#if defined(__APPLE__) || defined(__FreeBSD__) || defined(__OpenBSD__) || \ + defined(__DragonFly__) || defined(__NetBSD__) + hp_jitfunc_memory = mmap(0, 4096 + 4095, PROT_READ | PROT_WRITE | PROT_EXEC, + MAP_PRIVATE | MAP_ANON, 0, 0); +#else + hp_jitfunc_memory = mmap(0, 4096 + 4095, PROT_READ | PROT_WRITE | PROT_EXEC, + MAP_PRIVATE | MAP_ANONYMOUS, 0, 0); +#endif + if(hp_jitfunc_memory == MAP_FAILED) + hp_jitfunc_memory = NULL; +#endif + hp_jitfunc_allocated = 1; + if (hp_jitfunc_memory == NULL) + { + hp_jitfunc_allocated = 0; + hp_jitfunc_memory = malloc(4096 + 4095); + } + hp_jitfunc = (v4_random_math_JIT_func)((size_t)(hp_jitfunc_memory + 4095) & ~4095); +#if !(defined(_MSC_VER) || defined(__MINGW32__)) + mprotect(hp_jitfunc, 4096, PROT_READ | PROT_WRITE | PROT_EXEC); +#endif } /** @@ -660,8 +819,22 @@ void slow_hash_free_state(void) #endif } + if(!hp_jitfunc_allocated) + free(hp_jitfunc_memory); + else + { +#if defined(_MSC_VER) || defined(__MINGW32__) + VirtualFree(hp_jitfunc_memory, 0, MEM_RELEASE); +#else + munmap(hp_jitfunc_memory, 4096 + 4095); +#endif + } + hp_state = NULL; hp_allocated = 0; + hp_jitfunc = NULL; + hp_jitfunc_memory = NULL; + hp_jitfunc_allocated = 0; } /** @@ -694,7 +867,7 @@ void slow_hash_free_state(void) * @param length the length in bytes of the data * @param hash a pointer to a buffer in which the final 256 bit hash will be stored */ -void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed) +void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed, uint64_t height) { RDATA_ALIGN16 uint8_t expandedKey[240]; /* These buffers are aligned to use later with SSE functions */ @@ -730,6 +903,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int VARIANT1_INIT64(); VARIANT2_INIT64(); + VARIANT4_RANDOM_MATH_INIT(); /* CryptoNight Step 2: Iteratively encrypt the results from Keccak to fill * the 2MB large random access buffer. @@ -901,6 +1075,7 @@ union cn_slow_hash_state p = U64(&hp_state[j]); \ b[0] = p[0]; b[1] = p[1]; \ VARIANT2_PORTABLE_INTEGER_MATH(b, c); \ + VARIANT4_RANDOM_MATH(a, b, r, &_b, &_b1); \ __mul(); \ VARIANT2_2(); \ VARIANT2_SHUFFLE_ADD_NEON(hp_state, j); \ @@ -1063,7 +1238,7 @@ STATIC INLINE void aligned_free(void *ptr) } #endif /* FORCE_USE_HEAP */ -void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed) +void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed, uint64_t height) { RDATA_ALIGN16 uint8_t expandedKey[240]; @@ -1100,6 +1275,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int VARIANT1_INIT64(); VARIANT2_INIT64(); + VARIANT4_RANDOM_MATH_INIT(); /* CryptoNight Step 2: Iteratively encrypt the results from Keccak to fill * the 2MB large random access buffer. @@ -1278,10 +1454,11 @@ STATIC INLINE void xor_blocks(uint8_t* a, const uint8_t* b) U64(a)[1] ^= U64(b)[1]; } -void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed) +void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed, uint64_t height) { uint8_t text[INIT_SIZE_BYTE]; uint8_t a[AES_BLOCK_SIZE]; + uint8_t a1[AES_BLOCK_SIZE]; uint8_t b[AES_BLOCK_SIZE * 2]; uint8_t c[AES_BLOCK_SIZE]; uint8_t c1[AES_BLOCK_SIZE]; @@ -1317,6 +1494,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int VARIANT1_INIT64(); VARIANT2_INIT64(); + VARIANT4_RANDOM_MATH_INIT(); // use aligned data memcpy(expandedKey, aes_ctx->key->exp_data, aes_ctx->key->exp_data_len); @@ -1340,10 +1518,10 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int // Iteration 1 j = state_index(a); p = &long_state[j]; - aesb_single_round(p, p, a); - copy_block(c1, p); + aesb_single_round(p, c1, a); - VARIANT2_PORTABLE_SHUFFLE_ADD(long_state, j); + VARIANT2_PORTABLE_SHUFFLE_ADD(c1, a, long_state, j); + copy_block(p, c1); xor_blocks(p, b); VARIANT1_1(p); @@ -1352,13 +1530,15 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int p = &long_state[j]; copy_block(c, p); + copy_block(a1, a); VARIANT2_PORTABLE_INTEGER_MATH(c, c1); + VARIANT4_RANDOM_MATH(a1, c, r, b, b + AES_BLOCK_SIZE); mul(c1, c, d); VARIANT2_2_PORTABLE(); - VARIANT2_PORTABLE_SHUFFLE_ADD(long_state, j); - sum_half_blocks(a, d); - swap_blocks(a, c); - xor_blocks(a, c); + VARIANT2_PORTABLE_SHUFFLE_ADD(c1, a, long_state, j); + sum_half_blocks(a1, d); + swap_blocks(a1, c); + xor_blocks(a1, c); VARIANT1_2(U64(c) + 1); copy_block(p, c); @@ -1366,6 +1546,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int copy_block(b + AES_BLOCK_SIZE, b); } copy_block(b, c1); + copy_block(a, a1); } memcpy(text, state.init, INIT_SIZE_BYTE); @@ -1476,7 +1657,7 @@ union cn_slow_hash_state { }; #pragma pack(pop) -void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed) { +void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int prehashed, uint64_t height) { #ifndef FORCE_USE_HEAP uint8_t long_state[MEMORY]; #else @@ -1486,6 +1667,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int union cn_slow_hash_state state; uint8_t text[INIT_SIZE_BYTE]; uint8_t a[AES_BLOCK_SIZE]; + uint8_t a1[AES_BLOCK_SIZE]; uint8_t b[AES_BLOCK_SIZE * 2]; uint8_t c1[AES_BLOCK_SIZE]; uint8_t c2[AES_BLOCK_SIZE]; @@ -1505,6 +1687,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int VARIANT1_PORTABLE_INIT(); VARIANT2_PORTABLE_INIT(); + VARIANT4_RANDOM_MATH_INIT(); oaes_key_import_data(aes_ctx, aes_key, AES_KEY_SIZE); for (i = 0; i < MEMORY / INIT_SIZE_BYTE; i++) { @@ -1528,7 +1711,7 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int j = e2i(a, MEMORY / AES_BLOCK_SIZE) * AES_BLOCK_SIZE; copy_block(c1, &long_state[j]); aesb_single_round(c1, c1, a); - VARIANT2_PORTABLE_SHUFFLE_ADD(long_state, j); + VARIANT2_PORTABLE_SHUFFLE_ADD(c1, a, long_state, j); copy_block(&long_state[j], c1); xor_blocks(&long_state[j], b); assert(j == e2i(a, MEMORY / AES_BLOCK_SIZE) * AES_BLOCK_SIZE); @@ -1536,22 +1719,22 @@ void cn_slow_hash(const void *data, size_t length, char *hash, int variant, int /* Iteration 2 */ j = e2i(c1, MEMORY / AES_BLOCK_SIZE) * AES_BLOCK_SIZE; copy_block(c2, &long_state[j]); + copy_block(a1, a); VARIANT2_PORTABLE_INTEGER_MATH(c2, c1); + VARIANT4_RANDOM_MATH(a1, c2, r, b, b + AES_BLOCK_SIZE); mul(c1, c2, d); VARIANT2_2_PORTABLE(); - VARIANT2_PORTABLE_SHUFFLE_ADD(long_state, j); - swap_blocks(a, c1); - sum_half_blocks(c1, d); - swap_blocks(c1, c2); - xor_blocks(c1, c2); + VARIANT2_PORTABLE_SHUFFLE_ADD(c1, a, long_state, j); + sum_half_blocks(a1, d); + swap_blocks(a1, c2); + xor_blocks(a1, c2); VARIANT1_2(c2 + 8); copy_block(&long_state[j], c2); - assert(j == e2i(a, MEMORY / AES_BLOCK_SIZE) * AES_BLOCK_SIZE); if (variant >= 2) { copy_block(b + AES_BLOCK_SIZE, b); } - copy_block(b, a); - copy_block(a, c1); + copy_block(b, c1); + copy_block(a, a1); } memcpy(text, state.init, INIT_SIZE_BYTE); diff --git a/src/crypto/variant4_random_math.h b/src/crypto/variant4_random_math.h new file mode 100644 index 000000000..f3e41a001 --- /dev/null +++ b/src/crypto/variant4_random_math.h @@ -0,0 +1,441 @@ +#ifndef VARIANT4_RANDOM_MATH_H +#define VARIANT4_RANDOM_MATH_H + +// Register size can be configured to either 32 bit (uint32_t) or 64 bit (uint64_t) +typedef uint32_t v4_reg; + +enum V4_Settings +{ + // Generate code with minimal theoretical latency = 45 cycles, which is equivalent to 15 multiplications + TOTAL_LATENCY = 15 * 3, + + // Always generate at least 60 instructions + NUM_INSTRUCTIONS_MIN = 60, + + // Never generate more than 70 instructions (final RET instruction doesn't count here) + NUM_INSTRUCTIONS_MAX = 70, + + // Available ALUs for MUL + // Modern CPUs typically have only 1 ALU which can do multiplications + ALU_COUNT_MUL = 1, + + // Total available ALUs + // Modern CPUs have 4 ALUs, but we use only 3 because random math executes together with other main loop code + ALU_COUNT = 3, +}; + +enum V4_InstructionList +{ + MUL, // a*b + ADD, // a+b + C, C is an unsigned 32-bit constant + SUB, // a-b + ROR, // rotate right "a" by "b & 31" bits + ROL, // rotate left "a" by "b & 31" bits + XOR, // a^b + RET, // finish execution + V4_INSTRUCTION_COUNT = RET, +}; + +// V4_InstructionDefinition is used to generate code from random data +// Every random sequence of bytes is a valid code +// +// There are 9 registers in total: +// - 4 variable registers +// - 5 constant registers initialized from loop variables +// This is why dst_index is 2 bits +enum V4_InstructionDefinition +{ + V4_OPCODE_BITS = 3, + V4_DST_INDEX_BITS = 2, + V4_SRC_INDEX_BITS = 3, +}; + +struct V4_Instruction +{ + uint8_t opcode; + uint8_t dst_index; + uint8_t src_index; + uint32_t C; +}; + +#ifndef FORCEINLINE +#if defined(__GNUC__) +#define FORCEINLINE __attribute__((always_inline)) inline +#elif defined(_MSC_VER) +#define FORCEINLINE __forceinline +#else +#define FORCEINLINE inline +#endif +#endif + +#ifndef UNREACHABLE_CODE +#if defined(__GNUC__) +#define UNREACHABLE_CODE __builtin_unreachable() +#elif defined(_MSC_VER) +#define UNREACHABLE_CODE __assume(false) +#else +#define UNREACHABLE_CODE +#endif +#endif + +// Random math interpreter's loop is fully unrolled and inlined to achieve 100% branch prediction on CPU: +// every switch-case will point to the same destination on every iteration of Cryptonight main loop +// +// This is about as fast as it can get without using low-level machine code generation +static FORCEINLINE void v4_random_math(const struct V4_Instruction* code, v4_reg* r) +{ + enum + { + REG_BITS = sizeof(v4_reg) * 8, + }; + +#define V4_EXEC(i) \ + { \ + const struct V4_Instruction* op = code + i; \ + const v4_reg src = r[op->src_index]; \ + v4_reg* dst = r + op->dst_index; \ + switch (op->opcode) \ + { \ + case MUL: \ + *dst *= src; \ + break; \ + case ADD: \ + *dst += src + op->C; \ + break; \ + case SUB: \ + *dst -= src; \ + break; \ + case ROR: \ + { \ + const uint32_t shift = src % REG_BITS; \ + *dst = (*dst >> shift) | (*dst << ((REG_BITS - shift) % REG_BITS)); \ + } \ + break; \ + case ROL: \ + { \ + const uint32_t shift = src % REG_BITS; \ + *dst = (*dst << shift) | (*dst >> ((REG_BITS - shift) % REG_BITS)); \ + } \ + break; \ + case XOR: \ + *dst ^= src; \ + break; \ + case RET: \ + return; \ + default: \ + UNREACHABLE_CODE; \ + break; \ + } \ + } + +#define V4_EXEC_10(j) \ + V4_EXEC(j + 0) \ + V4_EXEC(j + 1) \ + V4_EXEC(j + 2) \ + V4_EXEC(j + 3) \ + V4_EXEC(j + 4) \ + V4_EXEC(j + 5) \ + V4_EXEC(j + 6) \ + V4_EXEC(j + 7) \ + V4_EXEC(j + 8) \ + V4_EXEC(j + 9) + + // Generated program can have 60 + a few more (usually 2-3) instructions to achieve required latency + // I've checked all block heights < 10,000,000 and here is the distribution of program sizes: + // + // 60 27960 + // 61 105054 + // 62 2452759 + // 63 5115997 + // 64 1022269 + // 65 1109635 + // 66 153145 + // 67 8550 + // 68 4529 + // 69 102 + + // Unroll 70 instructions here + V4_EXEC_10(0); // instructions 0-9 + V4_EXEC_10(10); // instructions 10-19 + V4_EXEC_10(20); // instructions 20-29 + V4_EXEC_10(30); // instructions 30-39 + V4_EXEC_10(40); // instructions 40-49 + V4_EXEC_10(50); // instructions 50-59 + V4_EXEC_10(60); // instructions 60-69 + +#undef V4_EXEC_10 +#undef V4_EXEC +} + +// If we don't have enough data available, generate more +static FORCEINLINE void check_data(size_t* data_index, const size_t bytes_needed, int8_t* data, const size_t data_size) +{ + if (*data_index + bytes_needed > data_size) + { + hash_extra_blake(data, data_size, (char*) data); + *data_index = 0; + } +} + +// Generates as many random math operations as possible with given latency and ALU restrictions +// "code" array must have space for NUM_INSTRUCTIONS_MAX+1 instructions +static inline int v4_random_math_init(struct V4_Instruction* code, const uint64_t height) +{ + // MUL is 3 cycles, 3-way addition and rotations are 2 cycles, SUB/XOR are 1 cycle + // These latencies match real-life instruction latencies for Intel CPUs starting from Sandy Bridge and up to Skylake/Coffee lake + // + // AMD Ryzen has the same latencies except 1-cycle ROR/ROL, so it'll be a bit faster than Intel Sandy Bridge and newer processors + // Surprisingly, Intel Nehalem also has 1-cycle ROR/ROL, so it'll also be faster than Intel Sandy Bridge and newer processors + // AMD Bulldozer has 4 cycles latency for MUL (slower than Intel) and 1 cycle for ROR/ROL (faster than Intel), so average performance will be the same + // Source: https://www.agner.org/optimize/instruction_tables.pdf + const int op_latency[V4_INSTRUCTION_COUNT] = { 3, 2, 1, 2, 2, 1 }; + + // Instruction latencies for theoretical ASIC implementation + const int asic_op_latency[V4_INSTRUCTION_COUNT] = { 3, 1, 1, 1, 1, 1 }; + + // Available ALUs for each instruction + const int op_ALUs[V4_INSTRUCTION_COUNT] = { ALU_COUNT_MUL, ALU_COUNT, ALU_COUNT, ALU_COUNT, ALU_COUNT, ALU_COUNT }; + + int8_t data[32]; + memset(data, 0, sizeof(data)); + uint64_t tmp = SWAP64LE(height); + memcpy(data, &tmp, sizeof(uint64_t)); + data[20] = -38; // change seed + + // Set data_index past the last byte in data + // to trigger full data update with blake hash + // before we start using it + size_t data_index = sizeof(data); + + int code_size; + + // There is a small chance (1.8%) that register R8 won't be used in the generated program + // So we keep track of it and try again if it's not used + bool r8_used; + do { + int latency[9]; + int asic_latency[9]; + + // Tracks previous instruction and value of the source operand for registers R0-R3 throughout code execution + // byte 0: current value of the destination register + // byte 1: instruction opcode + // byte 2: current value of the source register + // + // Registers R4-R8 are constant and are treated as having the same value because when we do + // the same operation twice with two constant source registers, it can be optimized into a single operation + uint32_t inst_data[9] = { 0, 1, 2, 3, 0xFFFFFF, 0xFFFFFF, 0xFFFFFF, 0xFFFFFF, 0xFFFFFF }; + + bool alu_busy[TOTAL_LATENCY + 1][ALU_COUNT]; + bool is_rotation[V4_INSTRUCTION_COUNT]; + bool rotated[4]; + int rotate_count = 0; + + memset(latency, 0, sizeof(latency)); + memset(asic_latency, 0, sizeof(asic_latency)); + memset(alu_busy, 0, sizeof(alu_busy)); + memset(is_rotation, 0, sizeof(is_rotation)); + memset(rotated, 0, sizeof(rotated)); + is_rotation[ROR] = true; + is_rotation[ROL] = true; + + int num_retries = 0; + code_size = 0; + + int total_iterations = 0; + r8_used = false; + + // Generate random code to achieve minimal required latency for our abstract CPU + // Try to get this latency for all 4 registers + while (((latency[0] < TOTAL_LATENCY) || (latency[1] < TOTAL_LATENCY) || (latency[2] < TOTAL_LATENCY) || (latency[3] < TOTAL_LATENCY)) && (num_retries < 64)) + { + // Fail-safe to guarantee loop termination + ++total_iterations; + if (total_iterations > 256) + break; + + check_data(&data_index, 1, data, sizeof(data)); + + const uint8_t c = ((uint8_t*)data)[data_index++]; + + // MUL = opcodes 0-2 + // ADD = opcode 3 + // SUB = opcode 4 + // ROR/ROL = opcode 5, shift direction is selected randomly + // XOR = opcodes 6-7 + uint8_t opcode = c & ((1 << V4_OPCODE_BITS) - 1); + if (opcode == 5) + { + check_data(&data_index, 1, data, sizeof(data)); + opcode = (data[data_index++] >= 0) ? ROR : ROL; + } + else if (opcode >= 6) + { + opcode = XOR; + } + else + { + opcode = (opcode <= 2) ? MUL : (opcode - 2); + } + + uint8_t dst_index = (c >> V4_OPCODE_BITS) & ((1 << V4_DST_INDEX_BITS) - 1); + uint8_t src_index = (c >> (V4_OPCODE_BITS + V4_DST_INDEX_BITS)) & ((1 << V4_SRC_INDEX_BITS) - 1); + + const int a = dst_index; + int b = src_index; + + // Don't do ADD/SUB/XOR with the same register + if (((opcode == ADD) || (opcode == SUB) || (opcode == XOR)) && (a == b)) + { + // Use register R8 as source instead + b = 8; + src_index = 8; + } + + // Don't do rotation with the same destination twice because it's equal to a single rotation + if (is_rotation[opcode] && rotated[a]) + { + continue; + } + + // Don't do the same instruction (except MUL) with the same source value twice because all other cases can be optimized: + // 2xADD(a, b, C) = ADD(a, b*2, C1+C2), same for SUB and rotations + // 2xXOR(a, b) = NOP + if ((opcode != MUL) && ((inst_data[a] & 0xFFFF00) == (opcode << 8) + ((inst_data[b] & 255) << 16))) + { + continue; + } + + // Find which ALU is available (and when) for this instruction + int next_latency = (latency[a] > latency[b]) ? latency[a] : latency[b]; + int alu_index = -1; + while (next_latency < TOTAL_LATENCY) + { + for (int i = op_ALUs[opcode] - 1; i >= 0; --i) + { + if (!alu_busy[next_latency][i]) + { + // ADD is implemented as two 1-cycle instructions on a real CPU, so do an additional availability check + if ((opcode == ADD) && alu_busy[next_latency + 1][i]) + { + continue; + } + + // Rotation can only start when previous rotation is finished, so do an additional availability check + if (is_rotation[opcode] && (next_latency < rotate_count * op_latency[opcode])) + { + continue; + } + + alu_index = i; + break; + } + } + if (alu_index >= 0) + { + break; + } + ++next_latency; + } + + // Don't generate instructions that leave some register unchanged for more than 7 cycles + if (next_latency > latency[a] + 7) + { + continue; + } + + next_latency += op_latency[opcode]; + + if (next_latency <= TOTAL_LATENCY) + { + if (is_rotation[opcode]) + { + ++rotate_count; + } + + // Mark ALU as busy only for the first cycle when it starts executing the instruction because ALUs are fully pipelined + alu_busy[next_latency - op_latency[opcode]][alu_index] = true; + latency[a] = next_latency; + + // ASIC is supposed to have enough ALUs to run as many independent instructions per cycle as possible, so latency calculation for ASIC is simple + asic_latency[a] = ((asic_latency[a] > asic_latency[b]) ? asic_latency[a] : asic_latency[b]) + asic_op_latency[opcode]; + + rotated[a] = is_rotation[opcode]; + + inst_data[a] = code_size + (opcode << 8) + ((inst_data[b] & 255) << 16); + + code[code_size].opcode = opcode; + code[code_size].dst_index = dst_index; + code[code_size].src_index = src_index; + code[code_size].C = 0; + + if (src_index == 8) + { + r8_used = true; + } + + if (opcode == ADD) + { + // ADD instruction is implemented as two 1-cycle instructions on a real CPU, so mark ALU as busy for the next cycle too + alu_busy[next_latency - op_latency[opcode] + 1][alu_index] = true; + + // ADD instruction requires 4 more random bytes for 32-bit constant "C" in "a = a + b + C" + check_data(&data_index, sizeof(uint32_t), data, sizeof(data)); + uint32_t t; + memcpy(&t, data + data_index, sizeof(uint32_t)); + code[code_size].C = SWAP32LE(t); + data_index += sizeof(uint32_t); + } + + ++code_size; + if (code_size >= NUM_INSTRUCTIONS_MIN) + { + break; + } + } + else + { + ++num_retries; + } + } + + // ASIC has more execution resources and can extract as much parallelism from the code as possible + // We need to add a few more MUL and ROR instructions to achieve minimal required latency for ASIC + // Get this latency for at least 1 of the 4 registers + const int prev_code_size = code_size; + while ((code_size < NUM_INSTRUCTIONS_MAX) && (asic_latency[0] < TOTAL_LATENCY) && (asic_latency[1] < TOTAL_LATENCY) && (asic_latency[2] < TOTAL_LATENCY) && (asic_latency[3] < TOTAL_LATENCY)) + { + int min_idx = 0; + int max_idx = 0; + for (int i = 1; i < 4; ++i) + { + if (asic_latency[i] < asic_latency[min_idx]) min_idx = i; + if (asic_latency[i] > asic_latency[max_idx]) max_idx = i; + } + + const uint8_t pattern[3] = { ROR, MUL, MUL }; + const uint8_t opcode = pattern[(code_size - prev_code_size) % 3]; + latency[min_idx] = latency[max_idx] + op_latency[opcode]; + asic_latency[min_idx] = asic_latency[max_idx] + asic_op_latency[opcode]; + + code[code_size].opcode = opcode; + code[code_size].dst_index = min_idx; + code[code_size].src_index = max_idx; + code[code_size].C = 0; + ++code_size; + } + + // There is ~98.15% chance that loop condition is false, so this loop will execute only 1 iteration most of the time + // It never does more than 4 iterations for all block heights < 10,000,000 + } while (!r8_used || (code_size < NUM_INSTRUCTIONS_MIN) || (code_size > NUM_INSTRUCTIONS_MAX)); + + // It's guaranteed that NUM_INSTRUCTIONS_MIN <= code_size <= NUM_INSTRUCTIONS_MAX here + // Add final instruction to stop the interpreter + code[code_size].opcode = RET; + code[code_size].dst_index = 0; + code[code_size].src_index = 0; + code[code_size].C = 0; + + return code_size; +} + +#endif diff --git a/src/cryptonote_basic/cryptonote_boost_serialization.h b/src/cryptonote_basic/cryptonote_boost_serialization.h index a4228b849..d1e321994 100644 --- a/src/cryptonote_basic/cryptonote_boost_serialization.h +++ b/src/cryptonote_basic/cryptonote_boost_serialization.h @@ -45,8 +45,6 @@ #include "ringct/rctTypes.h" #include "ringct/rctOps.h" -BOOST_CLASS_VERSION(rct::ecdhTuple, 1) - //namespace cryptonote { namespace boost { @@ -249,19 +247,8 @@ namespace boost template <class Archive> inline void serialize(Archive &a, rct::ecdhTuple &x, const boost::serialization::version_type ver) { - if (ver < 1) - { - a & x.mask; - a & x.amount; - return; - } - crypto::hash8 &amount = (crypto::hash8&)x.amount; - if (!Archive::is_saving::value) - { - memset(&x.mask, 0, sizeof(x.mask)); - memset(&x.amount, 0, sizeof(x.amount)); - } - a & amount; + a & x.mask; + a & x.amount; } template <class Archive> diff --git a/src/cryptonote_basic/cryptonote_format_utils.cpp b/src/cryptonote_basic/cryptonote_format_utils.cpp index f6daaab95..10fb5444c 100644 --- a/src/cryptonote_basic/cryptonote_format_utils.cpp +++ b/src/cryptonote_basic/cryptonote_format_utils.cpp @@ -1174,7 +1174,7 @@ namespace cryptonote } blobdata bd = get_block_hashing_blob(b); const int cn_variant = b.major_version >= 7 ? b.major_version - 6 : 0; - crypto::cn_slow_hash(bd.data(), bd.size(), res, cn_variant); + crypto::cn_slow_hash(bd.data(), bd.size(), res, cn_variant, height); return true; } //--------------------------------------------------------------- diff --git a/src/cryptonote_basic/hardfork.cpp b/src/cryptonote_basic/hardfork.cpp index 447d79aee..d1d836fcb 100644 --- a/src/cryptonote_basic/hardfork.cpp +++ b/src/cryptonote_basic/hardfork.cpp @@ -332,7 +332,7 @@ int HardFork::get_voted_fork_index(uint64_t height) const { CRITICAL_REGION_LOCAL(lock); uint32_t accumulated_votes = 0; - for (unsigned int n = heights.size() - 1; n > current_fork_index; --n) { + for (int n = heights.size() - 1; n >= 0; --n) { uint8_t v = heights[n].version; accumulated_votes += last_versions[v]; uint32_t threshold = (window_size * heights[n].threshold + 99) / 100; diff --git a/src/cryptonote_basic/miner.cpp b/src/cryptonote_basic/miner.cpp index f4de2ed7e..d29a3dcd6 100644 --- a/src/cryptonote_basic/miner.cpp +++ b/src/cryptonote_basic/miner.cpp @@ -73,6 +73,9 @@ #undef MONERO_DEFAULT_LOG_CATEGORY #define MONERO_DEFAULT_LOG_CATEGORY "miner" +#define AUTODETECT_WINDOW 10 // seconds +#define AUTODETECT_GAIN_THRESHOLD 1.02f // 2% + using namespace epee; #include "miner.h" @@ -108,6 +111,7 @@ namespace cryptonote m_starter_nonce(0), m_last_hr_merge_time(0), m_hashes(0), + m_total_hashes(0), m_do_print_hashrate(false), m_do_mining(false), m_current_hash_rate(0), @@ -178,7 +182,12 @@ namespace cryptonote merge_hr(); return true; }); - + + m_autodetect_interval.do_call([&](){ + update_autodetection(); + return true; + }); + return true; } //----------------------------------------------------------------------------------------------------- @@ -209,6 +218,60 @@ namespace cryptonote m_hashes = 0; } //----------------------------------------------------------------------------------------------------- + void miner::update_autodetection() + { + if (m_threads_autodetect.empty()) + return; + + uint64_t now = epee::misc_utils::get_ns_count(); + uint64_t dt = now - m_threads_autodetect.back().first; + if (dt < AUTODETECT_WINDOW * 1000000000ull) + return; + + // work out how many more hashes we got + m_threads_autodetect.back().first = dt; + uint64_t dh = m_total_hashes - m_threads_autodetect.back().second; + m_threads_autodetect.back().second = dh; + float hs = dh / (dt / (float)1000000000); + MGINFO("Mining autodetection: " << m_threads_autodetect.size() << " threads: " << hs << " H/s"); + + // when we don't increase by at least 2%, stop, otherwise check next + // if N and N+1 have mostly the same hash rate, we want to "lighter" one + bool found = false; + if (m_threads_autodetect.size() > 1) + { + int previdx = m_threads_autodetect.size() - 2; + float previous_hs = m_threads_autodetect[previdx].second / (m_threads_autodetect[previdx].first / (float)1000000000); + if (previous_hs > 0 && hs / previous_hs < AUTODETECT_GAIN_THRESHOLD) + { + m_threads_total = m_threads_autodetect.size() - 1; + m_threads_autodetect.clear(); + MGINFO("Optimal number of threads seems to be " << m_threads_total); + found = true; + } + } + + if (!found) + { + // setup one more thread + m_threads_autodetect.push_back({now, m_total_hashes}); + m_threads_total = m_threads_autodetect.size(); + } + + // restart all threads + { + CRITICAL_REGION_LOCAL(m_threads_lock); + boost::interprocess::ipcdetail::atomic_write32(&m_stop, 1); + for(boost::thread& th: m_threads) + th.join(); + m_threads.clear(); + } + boost::interprocess::ipcdetail::atomic_write32(&m_stop, 0); + boost::interprocess::ipcdetail::atomic_write32(&m_thread_index, 0); + for(size_t i = 0; i != m_threads_total; i++) + m_threads.push_back(boost::thread(m_attrs, boost::bind(&miner::worker_thread, this))); + } + //----------------------------------------------------------------------------------------------------- void miner::init_options(boost::program_options::options_description& desc) { command_line::add_arg(desc, arg_extra_messages); @@ -297,6 +360,13 @@ namespace cryptonote { m_mine_address = adr; m_threads_total = static_cast<uint32_t>(threads_count); + if (threads_count == 0) + { + m_threads_autodetect.clear(); + m_threads_autodetect.push_back({epee::misc_utils::get_ns_count(), m_total_hashes}); + m_threads_total = 1; + } + m_attrs = attrs; m_starter_nonce = crypto::rand<uint32_t>(); CRITICAL_REGION_LOCAL(m_threads_lock); if(is_mining()) @@ -318,12 +388,15 @@ namespace cryptonote set_is_background_mining_enabled(do_background); set_ignore_battery(ignore_battery); - for(size_t i = 0; i != threads_count; i++) + for(size_t i = 0; i != m_threads_total; i++) { m_threads.push_back(boost::thread(attrs, boost::bind(&miner::worker_thread, this))); } - LOG_PRINT_L0("Mining has started with " << threads_count << " threads, good luck!" ); + if (threads_count == 0) + MINFO("Mining has started, autodetecting optimal number of threads, good luck!" ); + else + MINFO("Mining has started with " << threads_count << " threads, good luck!" ); if( get_is_background_mining_enabled() ) { @@ -360,7 +433,7 @@ namespace cryptonote if (!is_mining()) { - MDEBUG("Not mining - nothing to stop" ); + MTRACE("Not mining - nothing to stop" ); return true; } @@ -381,6 +454,7 @@ namespace cryptonote MINFO("Mining has been stopped, " << m_threads.size() << " finished" ); m_threads.clear(); + m_threads_autodetect.clear(); return true; } //----------------------------------------------------------------------------------------------------- @@ -506,6 +580,7 @@ namespace cryptonote } nonce+=m_threads_total; ++m_hashes; + ++m_total_hashes; } slow_hash_free_state(); MGINFO("Miner thread stopped ["<< th_local_index << "]"); diff --git a/src/cryptonote_basic/miner.h b/src/cryptonote_basic/miner.h index e16d9f3b8..d4cdb2363 100644 --- a/src/cryptonote_basic/miner.h +++ b/src/cryptonote_basic/miner.h @@ -103,6 +103,7 @@ namespace cryptonote bool worker_thread(); bool request_block_template(); void merge_hr(); + void update_autodetection(); struct miner_config { @@ -132,16 +133,20 @@ namespace cryptonote account_public_address m_mine_address; epee::math_helper::once_a_time_seconds<5> m_update_block_template_interval; epee::math_helper::once_a_time_seconds<2> m_update_merge_hr_interval; + epee::math_helper::once_a_time_seconds<1> m_autodetect_interval; std::vector<blobdata> m_extra_messages; miner_config m_config; std::string m_config_folder_path; std::atomic<uint64_t> m_last_hr_merge_time; std::atomic<uint64_t> m_hashes; + std::atomic<uint64_t> m_total_hashes; std::atomic<uint64_t> m_current_hash_rate; epee::critical_section m_last_hash_rates_lock; std::list<uint64_t> m_last_hash_rates; bool m_do_print_hashrate; bool m_do_mining; + std::vector<std::pair<uint64_t, uint64_t>> m_threads_autodetect; + boost::thread::attributes m_attrs; // background mining stuffs .. diff --git a/src/cryptonote_config.h b/src/cryptonote_config.h index 4f652cd42..956cc76aa 100644 --- a/src/cryptonote_config.h +++ b/src/cryptonote_config.h @@ -59,6 +59,8 @@ #define CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V2 60000 //size of block (bytes) after which reward for block calculated using block size #define CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V1 20000 //size of block (bytes) after which reward for block calculated using block size - before first fork #define CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 300000 //size of block (bytes) after which reward for block calculated using block size - second change, from v5 +#define CRYPTONOTE_LONG_TERM_BLOCK_WEIGHT_WINDOW_SIZE 100000 // size in blocks of the long term block weight median window +#define CRYPTONOTE_SHORT_TERM_BLOCK_WEIGHT_SURGE_FACTOR 50 #define CRYPTONOTE_COINBASE_BLOB_RESERVED_SIZE 600 #define CRYPTONOTE_DISPLAY_DECIMAL_POINT 12 // COIN - number of smallest units in one coin @@ -108,6 +110,7 @@ #define P2P_DEFAULT_PACKET_MAX_SIZE 50000000 //50000000 bytes maximum packet size #define P2P_DEFAULT_PEERS_IN_HANDSHAKE 250 #define P2P_DEFAULT_CONNECTION_TIMEOUT 5000 //5 seconds +#define P2P_DEFAULT_SOCKS_CONNECT_TIMEOUT 45 // seconds #define P2P_DEFAULT_PING_CONNECTION_TIMEOUT 2000 //2 seconds #define P2P_DEFAULT_INVOKE_TIMEOUT 60*2*1000 //2 minutes #define P2P_DEFAULT_HANDSHAKE_INVOKE_TIMEOUT 5000 //5 seconds @@ -142,6 +145,7 @@ #define HF_VERSION_ENFORCE_RCT 6 #define HF_VERSION_PER_BYTE_FEE 8 #define HF_VERSION_SMALLER_BP 10 +#define HF_VERSION_LONG_TERM_BLOCK_WEIGHT 10 #define PER_KB_FEE_QUANTIZATION_DECIMALS 8 diff --git a/src/cryptonote_core/blockchain.cpp b/src/cryptonote_core/blockchain.cpp index 41357e72e..8fc401851 100644 --- a/src/cryptonote_core/blockchain.cpp +++ b/src/cryptonote_core/blockchain.cpp @@ -115,6 +115,12 @@ static const struct { // version 9 starts from block 1686275, which is on or around the 19th of October, 2018. Fork time finalised on 2018-09-02. { 9, 1686275, 0, 1535889548 }, + + // version 10 starts from block 1788000, which is on or around the 9th of March, 2019. Fork time finalised on 2019-02-10. + { 10, 1788000, 0, 1549792439 }, + + // version 11 starts from block 1788720, which is on or around the 10th of March, 2019. Fork time finalised on 2019-02-15. + { 11, 1788720, 0, 1550225678 }, }; static const uint64_t mainnet_hard_fork_version_1_till = 1009826; @@ -139,6 +145,8 @@ static const struct { { 7, 1057027, 0, 1512211236 }, { 8, 1057058, 0, 1533211200 }, { 9, 1057778, 0, 1533297600 }, + { 10, 1154318, 0, 1550153694 }, + { 11, 1155038, 0, 1550225678 }, }; static const uint64_t testnet_hard_fork_version_1_till = 624633; @@ -160,12 +168,16 @@ static const struct { { 7, 37000, 0, 1521600000 }, { 8, 176456, 0, 1537821770 }, { 9, 177176, 0, 1537821771 }, + { 10, 269000, 0, 1550153694 }, + { 11, 269720, 0, 1550225678 }, }; //------------------------------------------------------------------ Blockchain::Blockchain(tx_memory_pool& tx_pool) : m_db(), m_tx_pool(tx_pool), m_hardfork(NULL), m_timestamps_and_difficulties_height(0), m_current_block_cumul_weight_limit(0), m_current_block_cumul_weight_median(0), m_enforce_dns_checkpoints(false), m_max_prepare_blocks_threads(4), m_db_sync_on_blocks(true), m_db_sync_threshold(1), m_db_sync_mode(db_async), m_db_default_sync(false), m_fast_sync(true), m_show_time_stats(false), m_sync_counter(0), m_bytes_to_sync(0), m_cancel(false), + m_long_term_block_weights_window(CRYPTONOTE_LONG_TERM_BLOCK_WEIGHT_WINDOW_SIZE), + m_long_term_effective_median_block_weight(0), m_difficulty_for_next_block_top_hash(crypto::null_hash), m_difficulty_for_next_block(1), m_btc_valid(false) @@ -435,9 +447,9 @@ bool Blockchain::init(BlockchainDB* db, const network_type nettype, bool offline uint64_t top_block_timestamp = m_db->get_top_block_timestamp(); uint64_t timestamp_diff = time(NULL) - top_block_timestamp; - // genesis block has no timestamp, could probably change it to have timestamp of 1341378000... + // genesis block has no timestamp, could probably change it to have timestamp of 1397818133... if(!top_block_timestamp) - timestamp_diff = time(NULL) - 1341378000; + timestamp_diff = time(NULL) - 1397818133; // create general purpose async service queue @@ -500,7 +512,11 @@ bool Blockchain::init(BlockchainDB* db, const network_type nettype, bool offline m_tx_pool.on_blockchain_dec(m_db->height()-1, get_tail_id()); } - update_next_cumulative_weight_limit(); + if (test_options && test_options->long_term_block_weight_window) + m_long_term_block_weights_window = test_options->long_term_block_weight_window; + + if (!update_next_cumulative_weight_limit()) + return false; return true; } //------------------------------------------------------------------ @@ -685,7 +701,7 @@ block Blockchain::pop_block_from_blockchain() m_blocks_txs_check.clear(); m_check_txin_table.clear(); - update_next_cumulative_weight_limit(); + CHECK_AND_ASSERT_THROW_MES(update_next_cumulative_weight_limit(), "Error updating next cumulative weight limit"); m_tx_pool.on_blockchain_dec(m_db->height()-1, get_tail_id()); invalidate_block_template_cache(); @@ -704,7 +720,8 @@ bool Blockchain::reset_and_set_genesis_block(const block& b) block_verification_context bvc = boost::value_initialized<block_verification_context>(); add_new_block(b, bvc); - update_next_cumulative_weight_limit(); + if (!update_next_cumulative_weight_limit()) + return false; return bvc.m_added_to_main_chain && !bvc.m_verifivation_failed; } //------------------------------------------------------------------ @@ -1049,6 +1066,7 @@ bool Blockchain::switch_to_alternative_blockchain(std::list<blocks_ext_by_hash:: } // if we're to keep the disconnected blocks, add them as alternates + const size_t discarded_blocks = disconnected_chain.size(); if(!discard_disconnected_chain) { //pushing old chain as alternative chain @@ -1076,7 +1094,7 @@ bool Blockchain::switch_to_alternative_blockchain(std::list<blocks_ext_by_hash:: std::shared_ptr<tools::Notify> reorg_notify = m_reorg_notify; if (reorg_notify) reorg_notify->notify("%s", std::to_string(split_height).c_str(), "%h", std::to_string(m_db->height()).c_str(), - "%n", std::to_string(m_db->height() - split_height).c_str(), NULL); + "%n", std::to_string(m_db->height() - split_height).c_str(), "%d", std::to_string(discarded_blocks).c_str(), NULL); MGINFO_GREEN("REORGANIZE SUCCESS! on height: " << split_height << ", new blockchain size: " << m_db->height()); return true; @@ -1202,7 +1220,7 @@ bool Blockchain::validate_miner_transaction(const block& b, size_t cumulative_bl } } - std::vector<size_t> last_blocks_weights; + std::vector<uint64_t> last_blocks_weights; get_last_n_blocks_weights(last_blocks_weights, CRYPTONOTE_REWARD_BLOCKS_WINDOW); if (!get_block_reward(epee::misc_utils::median(last_blocks_weights), cumulative_block_weight, already_generated_coins, base_reward, version)) { @@ -1237,7 +1255,7 @@ bool Blockchain::validate_miner_transaction(const block& b, size_t cumulative_bl } //------------------------------------------------------------------ // get the block weights of the last <count> blocks, and return by reference <sz>. -void Blockchain::get_last_n_blocks_weights(std::vector<size_t>& weights, size_t count) const +void Blockchain::get_last_n_blocks_weights(std::vector<uint64_t>& weights, size_t count) const { LOG_PRINT_L3("Blockchain::" << __func__); CRITICAL_REGION_LOCAL(m_blockchain_lock); @@ -2527,7 +2545,19 @@ bool Blockchain::check_tx_outputs(const transaction& tx, tx_verification_context if (tx.version >= 2) { if (tx.rct_signatures.type == rct::RCTTypeBulletproof2) { - MERROR_VER("Bulletproofs v2 are not allowed before v" << HF_VERSION_SMALLER_BP); + MERROR_VER("Ringct type " << (unsigned)rct::RCTTypeBulletproof2 << " is not allowed before v" << HF_VERSION_SMALLER_BP); + tvc.m_invalid_output = true; + return false; + } + } + } + + // from v11, allow only bulletproofs v2 + if (hf_version > HF_VERSION_SMALLER_BP) { + if (tx.version >= 2) { + if (tx.rct_signatures.type == rct::RCTTypeBulletproof) + { + MERROR_VER("Ringct type " << (unsigned)rct::RCTTypeBulletproof << " is not allowed from v" << (HF_VERSION_SMALLER_BP + 1)); tvc.m_invalid_output = true; return false; } @@ -2600,7 +2630,7 @@ bool Blockchain::expand_transaction_2(transaction &tx, const crypto::hash &tx_pr for (size_t n = 0; n < tx.vin.size(); ++n) rv.p.MGs[0].II[n] = rct::ki2rct(boost::get<txin_to_key>(tx.vin[n]).k_image); } - else if (rv.type == rct::RCTTypeSimple || rv.type == rct::RCTTypeBulletproof || rct::RCTTypeBulletproof2) + else if (rv.type == rct::RCTTypeSimple || rv.type == rct::RCTTypeBulletproof || rv.type == rct::RCTTypeBulletproof2) { CHECK_AND_ASSERT_MES(rv.p.MGs.size() == tx.vin.size(), false, "Bad MGs size"); for (size_t n = 0; n < tx.vin.size(); ++n) @@ -3083,6 +3113,7 @@ uint64_t Blockchain::get_dynamic_base_fee(uint64_t block_reward, size_t median_b bool Blockchain::check_fee(size_t tx_weight, uint64_t fee) const { const uint8_t version = get_current_hard_fork_version(); + const uint64_t blockchain_height = m_db->height(); uint64_t median = 0; uint64_t already_generated_coins = 0; @@ -3090,7 +3121,7 @@ bool Blockchain::check_fee(size_t tx_weight, uint64_t fee) const if (version >= HF_VERSION_DYNAMIC_FEE) { median = m_current_block_cumul_weight_limit / 2; - already_generated_coins = m_db->height() ? m_db->get_block_already_generated_coins(m_db->height() - 1) : 0; + already_generated_coins = blockchain_height ? m_db->get_block_already_generated_coins(blockchain_height - 1) : 0; if (!get_block_reward(median, 1, already_generated_coins, base_reward, version)) return false; } @@ -3098,7 +3129,8 @@ bool Blockchain::check_fee(size_t tx_weight, uint64_t fee) const uint64_t needed_fee; if (version >= HF_VERSION_PER_BYTE_FEE) { - uint64_t fee_per_byte = get_dynamic_base_fee(base_reward, median, version); + const bool use_long_term_median_in_fee = version >= HF_VERSION_LONG_TERM_BLOCK_WEIGHT; + uint64_t fee_per_byte = get_dynamic_base_fee(base_reward, use_long_term_median_in_fee ? m_long_term_effective_median_block_weight : median, version); MDEBUG("Using " << print_money(fee_per_byte) << "/byte fee"); needed_fee = tx_weight * fee_per_byte; // quantize fee up to 8 decimals @@ -3135,6 +3167,7 @@ bool Blockchain::check_fee(size_t tx_weight, uint64_t fee) const uint64_t Blockchain::get_dynamic_base_fee_estimate(uint64_t grace_blocks) const { const uint8_t version = get_current_hard_fork_version(); + const uint64_t db_height = m_db->height(); if (version < HF_VERSION_DYNAMIC_FEE) return FEE_PER_KB; @@ -3143,7 +3176,7 @@ uint64_t Blockchain::get_dynamic_base_fee_estimate(uint64_t grace_blocks) const grace_blocks = CRYPTONOTE_REWARD_BLOCKS_WINDOW - 1; const uint64_t min_block_weight = get_min_block_weight(version); - std::vector<size_t> weights; + std::vector<uint64_t> weights; get_last_n_blocks_weights(weights, CRYPTONOTE_REWARD_BLOCKS_WINDOW - grace_blocks); weights.reserve(grace_blocks); for (size_t i = 0; i < grace_blocks; ++i) @@ -3153,7 +3186,7 @@ uint64_t Blockchain::get_dynamic_base_fee_estimate(uint64_t grace_blocks) const if(median <= min_block_weight) median = min_block_weight; - uint64_t already_generated_coins = m_db->height() ? m_db->get_block_already_generated_coins(m_db->height() - 1) : 0; + uint64_t already_generated_coins = db_height ? m_db->get_block_already_generated_coins(db_height - 1) : 0; uint64_t base_reward; if (!get_block_reward(median, 1, already_generated_coins, base_reward, version)) { @@ -3161,7 +3194,8 @@ uint64_t Blockchain::get_dynamic_base_fee_estimate(uint64_t grace_blocks) const base_reward = BLOCK_REWARD_OVERESTIMATE; } - uint64_t fee = get_dynamic_base_fee(base_reward, median, version); + const bool use_long_term_median_in_fee = version >= HF_VERSION_LONG_TERM_BLOCK_WEIGHT; + uint64_t fee = get_dynamic_base_fee(base_reward, use_long_term_median_in_fee ? m_long_term_effective_median_block_weight : median, version); const bool per_byte = version < HF_VERSION_PER_BYTE_FEE; MDEBUG("Estimating " << grace_blocks << "-block fee at " << print_money(fee) << "/" << (per_byte ? "byte" : "kB")); return fee; @@ -3658,7 +3692,8 @@ leave: { try { - new_height = m_db->add_block(bl, block_weight, cumulative_difficulty, already_generated_coins, txs); + uint64_t long_term_block_weight = get_next_long_term_block_weight(block_weight); + new_height = m_db->add_block(bl, block_weight, long_term_block_weight, cumulative_difficulty, already_generated_coins, txs); } catch (const KEY_IMAGE_EXISTS& e) { @@ -3684,7 +3719,12 @@ leave: TIME_MEASURE_FINISH(addblock); // do this after updating the hard fork state since the weight limit may change due to fork - update_next_cumulative_weight_limit(); + if (!update_next_cumulative_weight_limit()) + { + MERROR("Failed to update next cumulative weight limit"); + pop_block_from_blockchain(); + return false; + } MINFO("+++++ BLOCK SUCCESSFULLY ADDED" << std::endl << "id:\t" << id << std::endl << "PoW:\t" << proof_of_work << std::endl << "HEIGHT " << new_height-1 << ", difficulty:\t" << current_diffic << std::endl << "block reward: " << print_money(fee_summary + base_reward) << "(" << print_money(base_reward) << " + " << print_money(fee_summary) << "), coinbase_weight: " << coinbase_weight << ", cumulative weight: " << cumulative_block_weight << ", " << block_processing_time << "(" << target_calculating_time << "/" << longhash_calculating_time << ")ms"); if(m_show_time_stats) @@ -3740,20 +3780,100 @@ bool Blockchain::check_blockchain_pruning() return m_db->check_pruning(); } //------------------------------------------------------------------ -bool Blockchain::update_next_cumulative_weight_limit() +uint64_t Blockchain::get_next_long_term_block_weight(uint64_t block_weight) const { - uint64_t full_reward_zone = get_min_block_weight(get_current_hard_fork_version()); + PERF_TIMER(get_next_long_term_block_weight); + + const uint64_t db_height = m_db->height(); + const uint64_t nblocks = std::min<uint64_t>(m_long_term_block_weights_window, db_height); + + const uint8_t hf_version = get_current_hard_fork_version(); + if (hf_version < HF_VERSION_LONG_TERM_BLOCK_WEIGHT) + return block_weight; + + std::vector<uint64_t> weights; + weights.resize(nblocks); + for (uint64_t h = 0; h < nblocks; ++h) + weights[h] = m_db->get_block_long_term_weight(db_height - nblocks + h); + uint64_t long_term_median = epee::misc_utils::median(weights); + uint64_t long_term_effective_median_block_weight = std::max<uint64_t>(CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5, long_term_median); + + uint64_t short_term_constraint = long_term_effective_median_block_weight + long_term_effective_median_block_weight * 2 / 5; + uint64_t long_term_block_weight = std::min<uint64_t>(block_weight, short_term_constraint); + + return long_term_block_weight; +} +//------------------------------------------------------------------ +bool Blockchain::update_next_cumulative_weight_limit(uint64_t *long_term_effective_median_block_weight) +{ + PERF_TIMER(update_next_cumulative_weight_limit); LOG_PRINT_L3("Blockchain::" << __func__); - std::vector<size_t> weights; - get_last_n_blocks_weights(weights, CRYPTONOTE_REWARD_BLOCKS_WINDOW); - uint64_t median = epee::misc_utils::median(weights); - m_current_block_cumul_weight_median = median; - if(median <= full_reward_zone) - median = full_reward_zone; + // when we reach this, the last hf version is not yet written to the db + const uint64_t db_height = m_db->height(); + const uint8_t hf_version = get_current_hard_fork_version(); + uint64_t full_reward_zone = get_min_block_weight(hf_version); + uint64_t long_term_block_weight; + + if (hf_version < HF_VERSION_LONG_TERM_BLOCK_WEIGHT) + { + std::vector<uint64_t> weights; + get_last_n_blocks_weights(weights, CRYPTONOTE_REWARD_BLOCKS_WINDOW); + m_current_block_cumul_weight_median = epee::misc_utils::median(weights); + long_term_block_weight = weights.back(); + } + else + { + const uint64_t block_weight = m_db->get_block_weight(db_height - 1); + + std::vector<uint64_t> weights, new_weights; + uint64_t long_term_median; + if (db_height == 1) + { + long_term_median = CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5; + } + else + { + uint64_t nblocks = std::min<uint64_t>(m_long_term_block_weights_window, db_height); + if (nblocks == db_height) + --nblocks; + weights.resize(nblocks); + for (uint64_t h = 0; h < nblocks; ++h) + weights[h] = m_db->get_block_long_term_weight(db_height - nblocks + h - 1); + new_weights = weights; + long_term_median = epee::misc_utils::median(weights); + } + + m_long_term_effective_median_block_weight = std::max<uint64_t>(CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5, long_term_median); + + uint64_t short_term_constraint = m_long_term_effective_median_block_weight + m_long_term_effective_median_block_weight * 2 / 5; + long_term_block_weight = std::min<uint64_t>(block_weight, short_term_constraint); + + if (new_weights.empty()) + new_weights.resize(1); + new_weights[0] = long_term_block_weight; + long_term_median = epee::misc_utils::median(new_weights); + m_long_term_effective_median_block_weight = std::max<uint64_t>(CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5, long_term_median); + short_term_constraint = m_long_term_effective_median_block_weight + m_long_term_effective_median_block_weight * 2 / 5; + + weights.clear(); + get_last_n_blocks_weights(weights, CRYPTONOTE_REWARD_BLOCKS_WINDOW); + + uint64_t short_term_median = epee::misc_utils::median(weights); + uint64_t effective_median_block_weight = std::min<uint64_t>(std::max<uint64_t>(CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5, short_term_median), CRYPTONOTE_SHORT_TERM_BLOCK_WEIGHT_SURGE_FACTOR * m_long_term_effective_median_block_weight); + + m_current_block_cumul_weight_median = effective_median_block_weight; + } + + if (m_current_block_cumul_weight_median <= full_reward_zone) + m_current_block_cumul_weight_median = full_reward_zone; + + m_current_block_cumul_weight_limit = m_current_block_cumul_weight_median * 2; + + if (long_term_effective_median_block_weight) + *long_term_effective_median_block_weight = m_long_term_effective_median_block_weight; - m_current_block_cumul_weight_limit = median*2; return true; } //------------------------------------------------------------------ diff --git a/src/cryptonote_core/blockchain.h b/src/cryptonote_core/blockchain.h index c742bec91..92aef1278 100644 --- a/src/cryptonote_core/blockchain.h +++ b/src/cryptonote_core/blockchain.h @@ -37,6 +37,7 @@ #include <boost/multi_index/global_fun.hpp> #include <boost/multi_index/hashed_index.hpp> #include <boost/multi_index/member.hpp> +#include <boost/circular_buffer.hpp> #include <atomic> #include <functional> #include <unordered_map> @@ -631,6 +632,13 @@ namespace cryptonote uint64_t get_current_cumulative_block_weight_limit() const; /** + * @brief gets the long term block weight for a new block + * + * @return the long term block weight + */ + uint64_t get_next_long_term_block_weight(uint64_t block_weight) const; + + /** * @brief gets the block weight median based on recent blocks (same window as for the limit) * * @return the median @@ -994,7 +1002,9 @@ namespace cryptonote */ void pop_blocks(uint64_t nblocks); +#ifndef IN_UNIT_TESTS private: +#endif // TODO: evaluate whether or not each of these typedefs are left over from blockchain_storage typedef std::unordered_map<crypto::hash, size_t> blocks_by_id_index; @@ -1047,6 +1057,8 @@ namespace cryptonote std::vector<uint64_t> m_timestamps; std::vector<difficulty_type> m_difficulties; uint64_t m_timestamps_and_difficulties_height; + uint64_t m_long_term_block_weights_window; + uint64_t m_long_term_effective_median_block_weight; epee::critical_section m_difficulty_lock; crypto::hash m_difficulty_for_next_block_top_hash; @@ -1280,7 +1292,7 @@ namespace cryptonote * @param sz return-by-reference the list of weights * @param count the number of blocks to get weights for */ - void get_last_n_blocks_weights(std::vector<size_t>& weights, size_t count) const; + void get_last_n_blocks_weights(std::vector<uint64_t>& weights, size_t count) const; /** * @brief checks if a transaction is unlocked (its outputs spendable) @@ -1379,9 +1391,11 @@ namespace cryptonote /** * @brief calculate the block weight limit for the next block to be added * + * @param long_term_effective_median_block_weight optionally return that value + * * @return true */ - bool update_next_cumulative_weight_limit(); + bool update_next_cumulative_weight_limit(uint64_t *long_term_effective_median_block_weight = NULL); void return_tx_to_pool(std::vector<transaction> &txs); /** diff --git a/src/cryptonote_core/cryptonote_core.cpp b/src/cryptonote_core/cryptonote_core.cpp index 1513bb750..599f42774 100644 --- a/src/cryptonote_core/cryptonote_core.cpp +++ b/src/cryptonote_core/cryptonote_core.cpp @@ -188,8 +188,21 @@ namespace cryptonote static const command_line::arg_descriptor<std::string> arg_reorg_notify = { "reorg-notify" , "Run a program for each reorg, '%s' will be replaced by the split height, " - "'%h' will be replaced by the new blockchain height, and '%n' will be " - "replaced by the number of new blocks in the new chain" + "'%h' will be replaced by the new blockchain height, '%n' will be " + "replaced by the number of new blocks in the new chain, and '%d' will be " + "replaced by the number of blocks discarded from the old chain" + , "" + }; + static const command_line::arg_descriptor<std::string> arg_block_rate_notify = { + "block-rate-notify" + , "Run a program when the block rate undergoes large fluctuations. This might " + "be a sign of large amounts of hash rate going on and off the Monero network, " + "and thus be of potential interest in predicting attacks. %t will be replaced " + "by the number of minutes for the observation window, %b by the number of " + "blocks observed within that window, and %e by the number of blocks that was " + "expected in that window. It is suggested that this notification is used to " + "automatically increase the number of confirmations required before a payment " + "is acted upon." , "" }; @@ -308,6 +321,7 @@ namespace cryptonote command_line::add_arg(desc, arg_block_notify); command_line::add_arg(desc, arg_prune_blockchain); command_line::add_arg(desc, arg_reorg_notify); + command_line::add_arg(desc, arg_block_rate_notify); miner::init_options(desc); BlockchainDB::init_options(desc); @@ -587,7 +601,7 @@ namespace cryptonote } catch (const std::exception &e) { - MERROR("Failed to parse block notify spec"); + MERROR("Failed to parse block notify spec: " << e.what()); } try @@ -597,12 +611,23 @@ namespace cryptonote } catch (const std::exception &e) { - MERROR("Failed to parse reorg notify spec"); + MERROR("Failed to parse reorg notify spec: " << e.what()); + } + + try + { + if (!command_line::is_arg_defaulted(vm, arg_block_rate_notify)) + m_block_rate_notify.reset(new tools::Notify(command_line::get_arg(vm, arg_block_rate_notify).c_str())); + } + catch (const std::exception &e) + { + MERROR("Failed to parse block rate notify spec: " << e.what()); } const std::pair<uint8_t, uint64_t> regtest_hard_forks[3] = {std::make_pair(1, 0), std::make_pair(Blockchain::get_hard_fork_heights(MAINNET).back().version, 1), std::make_pair(0, 0)}; const cryptonote::test_options regtest_test_options = { - regtest_hard_forks + regtest_hard_forks, + 0 }; const difficulty_type fixed_difficulty = command_line::get_arg(vm, arg_fixed_difficulty); r = m_blockchain_storage.init(db.release(), m_nettype, m_offline, regtest ? ®test_test_options : test_options, fixed_difficulty, get_checkpoints); @@ -1764,7 +1789,7 @@ namespace cryptonote const time_t now = time(NULL); const std::vector<time_t> timestamps = m_blockchain_storage.get_last_block_timestamps(60); - static const unsigned int seconds[] = { 5400, 1800, 600 }; + static const unsigned int seconds[] = { 5400, 3600, 1800, 1200, 600 }; for (size_t n = 0; n < sizeof(seconds)/sizeof(seconds[0]); ++n) { unsigned int b = 0; @@ -1775,6 +1800,14 @@ namespace cryptonote if (p < threshold) { MWARNING("There were " << b << " blocks in the last " << seconds[n] / 60 << " minutes, there might be large hash rate changes, or we might be partitioned, cut off from the Monero network or under attack. Or it could be just sheer bad luck."); + + std::shared_ptr<tools::Notify> block_rate_notify = m_block_rate_notify; + if (block_rate_notify) + { + auto expected = seconds[n] / DIFFICULTY_TARGET_V2; + block_rate_notify->notify("%t", std::to_string(seconds[n] / 60).c_str(), "%b", std::to_string(b).c_str(), "%e", std::to_string(expected).c_str(), NULL); + } + break; // no need to look further } } diff --git a/src/cryptonote_core/cryptonote_core.h b/src/cryptonote_core/cryptonote_core.h index 4810fc891..79d06662e 100644 --- a/src/cryptonote_core/cryptonote_core.h +++ b/src/cryptonote_core/cryptonote_core.h @@ -54,6 +54,7 @@ namespace cryptonote { struct test_options { const std::pair<uint8_t, uint64_t> *hard_forks; + const size_t long_term_block_weight_window; }; extern const command_line::arg_descriptor<std::string, false, true, 2> arg_data_dir; @@ -1061,6 +1062,8 @@ namespace cryptonote bool m_fluffy_blocks_enabled; bool m_offline; bool m_pad_transactions; + + std::shared_ptr<tools::Notify> m_block_rate_notify; }; } diff --git a/src/cryptonote_core/cryptonote_tx_utils.cpp b/src/cryptonote_core/cryptonote_tx_utils.cpp index 0a04e0d38..c138c7854 100644 --- a/src/cryptonote_core/cryptonote_tx_utils.cpp +++ b/src/cryptonote_core/cryptonote_tx_utils.cpp @@ -405,49 +405,12 @@ namespace cryptonote for(const tx_destination_entry& dst_entr: destinations) { CHECK_AND_ASSERT_MES(dst_entr.amount > 0 || tx.version > 1, false, "Destination with wrong amount: " << dst_entr.amount); - crypto::key_derivation derivation; crypto::public_key out_eph_public_key; - // make additional tx pubkey if necessary - keypair additional_txkey; - if (need_additional_txkeys) - { - additional_txkey.sec = additional_tx_keys[output_index]; - if (dst_entr.is_subaddress) - additional_txkey.pub = rct::rct2pk(hwdev.scalarmultKey(rct::pk2rct(dst_entr.addr.m_spend_public_key), rct::sk2rct(additional_txkey.sec))); - else - additional_txkey.pub = rct::rct2pk(hwdev.scalarmultBase(rct::sk2rct(additional_txkey.sec))); - } - - bool r; - if (change_addr && dst_entr.addr == *change_addr) - { - // sending change to yourself; derivation = a*R - r = hwdev.generate_key_derivation(txkey_pub, sender_account_keys.m_view_secret_key, derivation); - CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to generate_key_derivation(" << txkey_pub << ", " << sender_account_keys.m_view_secret_key << ")"); - } - else - { - // sending to the recipient; derivation = r*A (or s*C in the subaddress scheme) - r = hwdev.generate_key_derivation(dst_entr.addr.m_view_public_key, dst_entr.is_subaddress && need_additional_txkeys ? additional_txkey.sec : tx_key, derivation); - CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to generate_key_derivation(" << dst_entr.addr.m_view_public_key << ", " << (dst_entr.is_subaddress && need_additional_txkeys ? additional_txkey.sec : tx_key) << ")"); - } - - if (need_additional_txkeys) - { - additional_tx_public_keys.push_back(additional_txkey.pub); - } - - if (tx.version > 1) - { - crypto::secret_key scalar1; - hwdev.derivation_to_scalar(derivation, output_index, scalar1); - amount_keys.push_back(rct::sk2rct(scalar1)); - } - r = hwdev.derive_public_key(derivation, output_index, dst_entr.addr.m_spend_public_key, out_eph_public_key); - CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to derive_public_key(" << derivation << ", " << output_index << ", "<< dst_entr.addr.m_spend_public_key << ")"); - - hwdev.add_output_key_mapping(dst_entr.addr.m_view_public_key, dst_entr.addr.m_spend_public_key, dst_entr.is_subaddress, output_index, amount_keys.back(), out_eph_public_key); + hwdev.generate_output_ephemeral_keys(tx.version,sender_account_keys, txkey_pub, tx_key, + dst_entr, change_addr, output_index, + need_additional_txkeys, additional_tx_keys, + additional_tx_public_keys, amount_keys, out_eph_public_key); tx_out out; out.amount = dst_entr.amount; diff --git a/src/cryptonote_core/cryptonote_tx_utils.h b/src/cryptonote_core/cryptonote_tx_utils.h index 85061668b..ad3297951 100644 --- a/src/cryptonote_core/cryptonote_tx_utils.h +++ b/src/cryptonote_core/cryptonote_tx_utils.h @@ -97,6 +97,12 @@ namespace cryptonote bool construct_tx(const account_keys& sender_account_keys, std::vector<tx_source_entry> &sources, const std::vector<tx_destination_entry>& destinations, const boost::optional<cryptonote::account_public_address>& change_addr, const std::vector<uint8_t> &extra, transaction& tx, uint64_t unlock_time); bool construct_tx_with_tx_key(const account_keys& sender_account_keys, const std::unordered_map<crypto::public_key, subaddress_index>& subaddresses, std::vector<tx_source_entry>& sources, std::vector<tx_destination_entry>& destinations, const boost::optional<cryptonote::account_public_address>& change_addr, const std::vector<uint8_t> &extra, transaction& tx, uint64_t unlock_time, const crypto::secret_key &tx_key, const std::vector<crypto::secret_key> &additional_tx_keys, bool rct = false, const rct::RCTConfig &rct_config = { rct::RangeProofBorromean, 0 }, rct::multisig_out *msout = NULL, bool shuffle_outs = true); bool construct_tx_and_get_tx_key(const account_keys& sender_account_keys, const std::unordered_map<crypto::public_key, subaddress_index>& subaddresses, std::vector<tx_source_entry>& sources, std::vector<tx_destination_entry>& destinations, const boost::optional<cryptonote::account_public_address>& change_addr, const std::vector<uint8_t> &extra, transaction& tx, uint64_t unlock_time, crypto::secret_key &tx_key, std::vector<crypto::secret_key> &additional_tx_keys, bool rct = false, const rct::RCTConfig &rct_config = { rct::RangeProofBorromean, 0 }, rct::multisig_out *msout = NULL); + bool generate_output_ephemeral_keys(const size_t tx_version, const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, + crypto::public_key &out_eph_public_key) ; bool generate_genesis_block( block& bl diff --git a/src/cryptonote_core/tx_pool.h b/src/cryptonote_core/tx_pool.h index 670d70d77..8dd0337f0 100644 --- a/src/cryptonote_core/tx_pool.h +++ b/src/cryptonote_core/tx_pool.h @@ -61,7 +61,7 @@ namespace cryptonote class txCompare { public: - bool operator()(const tx_by_fee_and_receive_time_entry& a, const tx_by_fee_and_receive_time_entry& b) + bool operator()(const tx_by_fee_and_receive_time_entry& a, const tx_by_fee_and_receive_time_entry& b) const { // sort by greatest first, not least if (a.first.first > b.first.first) return true; diff --git a/src/cryptonote_protocol/block_queue.cpp b/src/cryptonote_protocol/block_queue.cpp index 672696944..716b0c8ba 100644 --- a/src/cryptonote_protocol/block_queue.cpp +++ b/src/cryptonote_protocol/block_queue.cpp @@ -240,7 +240,7 @@ std::pair<uint64_t, uint64_t> block_queue::reserve_span(uint64_t first_block_hei MDEBUG("reserve_span: early out: first_block_height " << first_block_height << ", last_block_height " << last_block_height << ", max_blocks " << max_blocks); return std::make_pair(0, 0); } - if (block_hashes.size() >= last_block_height) + if (block_hashes.size() > last_block_height) { MDEBUG("reserve_span: more block hashes than fit within last_block_height: " << block_hashes.size() << " and " << last_block_height); return std::make_pair(0, 0); diff --git a/src/cryptonote_protocol/cryptonote_protocol_defs.h b/src/cryptonote_protocol/cryptonote_protocol_defs.h index c49371d48..2d5d10d67 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_defs.h +++ b/src/cryptonote_protocol/cryptonote_protocol_defs.h @@ -48,6 +48,7 @@ namespace cryptonote bool incoming; bool localhost; bool local_ip; + bool ssl; std::string address; std::string host; diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.h b/src/cryptonote_protocol/cryptonote_protocol_handler.h index a1bd9171c..efd986b53 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.h +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.h @@ -173,15 +173,6 @@ namespace cryptonote //handler_response_blocks_now(blob.size()); // XXX return m_p2p->invoke_notify_to_peer(t_parameter::ID, epee::strspan<uint8_t>(blob), context); } - - template<class t_parameter> - bool relay_post_notify(typename t_parameter::request& arg, cryptonote_connection_context& exclude_context) - { - LOG_PRINT_L2("[" << epee::net_utils::print_connection_context_short(exclude_context) << "] post relay " << typeid(t_parameter).name() << " -->"); - std::string arg_buff; - epee::serialization::store_t_to_binary(arg, arg_buff); - return m_p2p->relay_notify_to_all(t_parameter::ID, epee::strspan<uint8_t>(arg_buff), exclude_context); - } }; } // namespace diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.inl b/src/cryptonote_protocol/cryptonote_protocol_handler.inl index 61a211094..8fada4e3c 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.inl +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.inl @@ -226,7 +226,7 @@ namespace cryptonote cnx.host = cntxt.m_remote_address.host_str(); cnx.ip = ""; cnx.port = ""; - if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { cnx.ip = cnx.host; cnx.port = std::to_string(cntxt.m_remote_address.as<epee::net_utils::ipv4_network_address>().port()); @@ -268,6 +268,7 @@ namespace cryptonote cnx.current_upload = cntxt.m_current_speed_up / 1024; cnx.connection_id = epee::string_tools::pod_to_hex(cntxt.m_connection_id); + cnx.ssl = cntxt.m_ssl; cnx.height = cntxt.m_remote_blockchain_height; cnx.pruning_seed = cntxt.m_pruning_seed; @@ -333,6 +334,13 @@ namespace cryptonote return true; } + // No chain synchronization over hidden networks (tor, i2p, etc.) + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + { + context.m_state = cryptonote_connection_context::state_normal; + return true; + } + if (hshd.current_height > target) { /* As I don't know if accessing hshd from core could be a good practice, @@ -2058,20 +2066,20 @@ skip: fluffy_arg.b.txs = fluffy_txs; // sort peers between fluffy ones and others - std::list<boost::uuids::uuid> fullConnections, fluffyConnections; + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> fullConnections, fluffyConnections; m_p2p->for_each_connection([this, &exclude_context, &fullConnections, &fluffyConnections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) { - if (peer_id && exclude_context.m_connection_id != context.m_connection_id) + if (peer_id && exclude_context.m_connection_id != context.m_connection_id && context.m_remote_address.get_zone() == epee::net_utils::zone::public_) { if(m_core.fluffy_blocks_enabled() && (support_flags & P2P_SUPPORT_FLAG_FLUFFY_BLOCKS)) { LOG_DEBUG_CC(context, "PEER SUPPORTS FLUFFY BLOCKS - RELAYING THIN/COMPACT WHATEVER BLOCK"); - fluffyConnections.push_back(context.m_connection_id); + fluffyConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } else { LOG_DEBUG_CC(context, "PEER DOESN'T SUPPORT FLUFFY BLOCKS - RELAYING FULL BLOCK"); - fullConnections.push_back(context.m_connection_id); + fullConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } } return true; @@ -2082,13 +2090,13 @@ skip: { std::string fluffyBlob; epee::serialization::store_t_to_binary(fluffy_arg, fluffyBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), fluffyConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), std::move(fluffyConnections)); } if (!fullConnections.empty()) { std::string fullBlob; epee::serialization::store_t_to_binary(arg, fullBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), fullConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), std::move(fullConnections)); } return true; @@ -2097,8 +2105,14 @@ skip: template<class t_core> bool t_cryptonote_protocol_handler<t_core>::relay_transactions(NOTIFY_NEW_TRANSACTIONS::request& arg, cryptonote_connection_context& exclude_context) { + const bool hide_tx_broadcast = + 1 < m_p2p->get_zone_count() && exclude_context.m_remote_address.get_zone() == epee::net_utils::zone::invalid; + + if (hide_tx_broadcast) + MDEBUG("Attempting to conceal origin of tx via anonymity network connection(s)"); + // no check for success, so tell core they're relayed unconditionally - const bool pad_transactions = m_core.pad_transactions(); + const bool pad_transactions = m_core.pad_transactions() || hide_tx_broadcast; size_t bytes = pad_transactions ? 9 /* header */ + 4 /* 1 + 'txs' */ + tools::get_varint_data(arg.txs.size()).size() : 0; for(auto tx_blob_it = arg.txs.begin(); tx_blob_it!=arg.txs.end(); ++tx_blob_it) { @@ -2131,7 +2145,30 @@ skip: // if the size of _ moved enough, we might lose byte in size encoding, we don't care } - return relay_post_notify<NOTIFY_NEW_TRANSACTIONS>(arg, exclude_context); + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections; + m_p2p->for_each_connection([hide_tx_broadcast, &exclude_context, &connections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) + { + const epee::net_utils::zone current_zone = context.m_remote_address.get_zone(); + const bool broadcast_to_peer = + peer_id && + (hide_tx_broadcast != bool(current_zone == epee::net_utils::zone::public_)) && + exclude_context.m_connection_id != context.m_connection_id; + + if (broadcast_to_peer) + connections.push_back({current_zone, context.m_connection_id}); + + return true; + }); + + if (connections.empty()) + MERROR("Transaction not relayed - no" << (hide_tx_broadcast ? " privacy": "") << " peers available"); + else + { + std::string fullBlob; + epee::serialization::store_t_to_binary(arg, fullBlob); + m_p2p->relay_notify_to_list(NOTIFY_NEW_TRANSACTIONS::ID, epee::strspan<uint8_t>(fullBlob), std::move(connections)); + } + return true; } //------------------------------------------------------------------------------------------------------------------------ template<class t_core> diff --git a/src/daemon/command_parser_executor.cpp b/src/daemon/command_parser_executor.cpp index 73f33a674..00d004970 100644 --- a/src/daemon/command_parser_executor.cpp +++ b/src/daemon/command_parser_executor.cpp @@ -302,7 +302,7 @@ bool t_command_parser_executor::start_mining(const std::vector<std::string>& arg { if(!args.size()) { - std::cout << "Please specify a wallet address to mine for: start_mining <addr> [<threads>]" << std::endl; + std::cout << "Please specify a wallet address to mine for: start_mining <addr> [<threads>|auto]" << std::endl; return true; } @@ -388,8 +388,15 @@ bool t_command_parser_executor::start_mining(const std::vector<std::string>& arg if(args.size() >= 2) { - bool ok = epee::string_tools::get_xtype_from_string(threads_count, args[1]); - threads_count = (ok && 0 < threads_count) ? threads_count : 1; + if (args[1] == "auto" || args[1] == "autodetect") + { + threads_count = 0; + } + else + { + bool ok = epee::string_tools::get_xtype_from_string(threads_count, args[1]); + threads_count = (ok && 0 < threads_count) ? threads_count : 1; + } } m_executor.start_mining(info.address, threads_count, nettype, do_background_mining, ignore_battery); diff --git a/src/daemon/command_server.cpp b/src/daemon/command_server.cpp index 96dea76b6..3e19bb42f 100644 --- a/src/daemon/command_server.cpp +++ b/src/daemon/command_server.cpp @@ -104,8 +104,8 @@ t_command_server::t_command_server( m_command_lookup.set_handler( "start_mining" , std::bind(&t_command_parser_executor::start_mining, &m_parser, p::_1) - , "start_mining <addr> [<threads>] [do_background_mining] [ignore_battery]" - , "Start mining for specified address. Defaults to 1 thread and no background mining." + , "start_mining <addr> [<threads>|auto] [do_background_mining] [ignore_battery]" + , "Start mining for specified address. Defaults to 1 thread and no background mining. Use \"auto\" to autodetect optimal number of threads." ); m_command_lookup.set_handler( "stop_mining" diff --git a/src/daemon/rpc_command_executor.cpp b/src/daemon/rpc_command_executor.cpp index 839350522..0a35dcef9 100644 --- a/src/daemon/rpc_command_executor.cpp +++ b/src/daemon/rpc_command_executor.cpp @@ -79,6 +79,7 @@ namespace { << "POW hash: " << header.pow_hash << std::endl << "block size: " << header.block_size << std::endl << "block weight: " << header.block_weight << std::endl + << "long term weight: " << header.long_term_weight << std::endl << "num txes: " << header.num_txes << std::endl << "reward: " << cryptonote::print_money(header.reward); } @@ -510,6 +511,7 @@ bool t_rpc_command_executor::print_connections() { } tools::msg_writer() << std::setw(30) << std::left << "Remote Host" + << std::setw(6) << "SSL" << std::setw(20) << "Peer id" << std::setw(20) << "Support Flags" << std::setw(30) << "Recv/Sent (inactive,sec)" @@ -529,6 +531,7 @@ bool t_rpc_command_executor::print_connections() { tools::msg_writer() //<< std::setw(30) << std::left << in_out << std::setw(30) << std::left << address + << std::setw(6) << (info.ssl ? "yes" : "no") << std::setw(20) << epee::string_tools::pad_string(info.peer_id, 16, '0', true) << std::setw(20) << info.support_flags << std::setw(30) << std::to_string(info.recv_count) + "(" + std::to_string(info.recv_idle_time) + ")/" + std::to_string(info.send_count) + "(" + std::to_string(info.send_idle_time) + ")" diff --git a/src/device/device.hpp b/src/device/device.hpp index bdb608907..408f64c8b 100644 --- a/src/device/device.hpp +++ b/src/device/device.hpp @@ -69,6 +69,7 @@ namespace cryptonote struct account_public_address; struct account_keys; struct subaddress_index; + struct tx_destination_entry; } namespace hw { @@ -211,9 +212,12 @@ namespace hw { virtual bool ecdhEncode(rct::ecdhTuple & unmasked, const rct::key & sharedSec, bool short_amount) = 0; virtual bool ecdhDecode(rct::ecdhTuple & masked, const rct::key & sharedSec, bool short_amount) = 0; - virtual bool add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const size_t real_output_index, - const rct::key &amount_key, const crypto::public_key &out_eph_public_key) = 0; - + virtual bool generate_output_ephemeral_keys(const size_t tx_version, const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, + crypto::public_key &out_eph_public_key) = 0; virtual bool mlsag_prehash(const std::string &blob, size_t inputs_size, size_t outputs_size, const rct::keyV &hashes, const rct::ctkeyV &outPk, rct::key &prehash) = 0; virtual bool mlsag_prepare(const rct::key &H, const rct::key &xx, rct::key &a, rct::key &aG, rct::key &aHP, rct::key &rvII) = 0; diff --git a/src/device/device_default.cpp b/src/device/device_default.cpp index cb2f4e266..fd15717a7 100644 --- a/src/device/device_default.cpp +++ b/src/device/device_default.cpp @@ -34,8 +34,10 @@ #include "int-util.h" #include "cryptonote_basic/account.h" #include "cryptonote_basic/subaddress_index.h" +#include "cryptonote_core/cryptonote_tx_utils.h" #include "ringct/rctOps.h" +#include "log.hpp" #define ENCRYPTED_PAYMENT_ID_TAIL 0x8d #define CHACHA8_KEY_TAIL 0x8c @@ -278,10 +280,55 @@ namespace hw { return true; } + bool device_default::generate_output_ephemeral_keys(const size_t tx_version, + const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, crypto::public_key &out_eph_public_key) { - bool device_default::add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const size_t real_output_index, - const rct::key &amount_key, const crypto::public_key &out_eph_public_key) { - return true; + crypto::key_derivation derivation; + + // make additional tx pubkey if necessary + cryptonote::keypair additional_txkey; + if (need_additional_txkeys) + { + additional_txkey.sec = additional_tx_keys[output_index]; + if (dst_entr.is_subaddress) + additional_txkey.pub = rct::rct2pk(rct::scalarmultKey(rct::pk2rct(dst_entr.addr.m_spend_public_key), rct::sk2rct(additional_txkey.sec))); + else + additional_txkey.pub = rct::rct2pk(rct::scalarmultBase(rct::sk2rct(additional_txkey.sec))); + } + + bool r; + if (change_addr && dst_entr.addr == *change_addr) + { + // sending change to yourself; derivation = a*R + r = generate_key_derivation(txkey_pub, sender_account_keys.m_view_secret_key, derivation); + CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to generate_key_derivation(" << txkey_pub << ", " << sender_account_keys.m_view_secret_key << ")"); + } + else + { + // sending to the recipient; derivation = r*A (or s*C in the subaddress scheme) + r = generate_key_derivation(dst_entr.addr.m_view_public_key, dst_entr.is_subaddress && need_additional_txkeys ? additional_txkey.sec : tx_key, derivation); + CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to generate_key_derivation(" << dst_entr.addr.m_view_public_key << ", " << (dst_entr.is_subaddress && need_additional_txkeys ? additional_txkey.sec : tx_key) << ")"); + } + + if (need_additional_txkeys) + { + additional_tx_public_keys.push_back(additional_txkey.pub); + } + + if (tx_version > 1) + { + crypto::secret_key scalar1; + derivation_to_scalar(derivation, output_index, scalar1); + amount_keys.push_back(rct::sk2rct(scalar1)); + } + r = derive_public_key(derivation, output_index, dst_entr.addr.m_spend_public_key, out_eph_public_key); + CHECK_AND_ASSERT_MES(r, false, "at creation outs: failed to derive_public_key(" << derivation << ", " << output_index << ", "<< dst_entr.addr.m_spend_public_key << ")"); + + return r; } bool device_default::encrypt_payment_id(crypto::hash8 &payment_id, const crypto::public_key &public_key, const crypto::secret_key &secret_key) { diff --git a/src/device/device_default.hpp b/src/device/device_default.hpp index 54d159b11..04b9b4234 100644 --- a/src/device/device_default.hpp +++ b/src/device/device_default.hpp @@ -114,9 +114,12 @@ namespace hw { bool ecdhEncode(rct::ecdhTuple & unmasked, const rct::key & sharedSec, bool short_amount) override; bool ecdhDecode(rct::ecdhTuple & masked, const rct::key & sharedSec, bool short_amount) override; - bool add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const size_t real_output_index, - const rct::key &amount_key, const crypto::public_key &out_eph_public_key) override; - + bool generate_output_ephemeral_keys(const size_t tx_version, const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, + crypto::public_key &out_eph_public_key) override; bool mlsag_prehash(const std::string &blob, size_t inputs_size, size_t outputs_size, const rct::keyV &hashes, const rct::ctkeyV &outPk, rct::key &prehash) override; bool mlsag_prepare(const rct::key &H, const rct::key &xx, rct::key &a, rct::key &aG, rct::key &aHP, rct::key &rvII) override; diff --git a/src/device/device_ledger.cpp b/src/device/device_ledger.cpp index 9daf62d37..1f91427f0 100644 --- a/src/device/device_ledger.cpp +++ b/src/device/device_ledger.cpp @@ -32,6 +32,7 @@ #include "ringct/rctOps.h" #include "cryptonote_basic/account.h" #include "cryptonote_basic/subaddress_index.h" +#include "cryptonote_core/cryptonote_tx_utils.h" #include <boost/thread/locks.hpp> #include <boost/thread/lock_guard.hpp> @@ -67,10 +68,12 @@ namespace hw { /* === Keymap ==== */ /* ===================================================================== */ - ABPkeys::ABPkeys(const rct::key& A, const rct::key& B, const bool is_subaddr, const size_t real_output_index, const rct::key& P, const rct::key& AK) { + ABPkeys::ABPkeys(const rct::key& A, const rct::key& B, const bool is_subaddr, const bool is_change, const bool need_additional_txkeys, const size_t real_output_index, const rct::key& P, const rct::key& AK) { Aout = A; Bout = B; is_subaddress = is_subaddr; + is_change_address = is_change; + additional_key = need_additional_txkeys; index = real_output_index; Pout = P; AKout = AK; @@ -80,6 +83,8 @@ namespace hw { Aout = keys.Aout; Bout = keys.Bout; is_subaddress = keys.is_subaddress; + is_change_address = keys.is_change_address; + additional_key = keys.additional_key; index = keys.index; Pout = keys.Pout; AKout = keys.AKout; @@ -137,6 +142,8 @@ namespace hw { static int device_id = 0; + #define PROTOCOL_VERSION 2 + #define INS_NONE 0x00 #define INS_RESET 0x02 @@ -168,6 +175,7 @@ namespace hw { #define INS_STEALTH 0x76 #define INS_BLIND 0x78 #define INS_UNBLIND 0x7A + #define INS_GEN_TXOUT_KEYS 0x7B #define INS_VALIDATE 0x7C #define INS_MLSAG 0x7E #define INS_CLOSE_TX 0x80 @@ -268,7 +276,7 @@ namespace hw { int device_ledger::set_command_header(unsigned char ins, unsigned char p1, unsigned char p2) { reset_buffer(); int offset = 0; - this->buffer_send[0] = 0x00; + this->buffer_send[0] = PROTOCOL_VERSION; this->buffer_send[1] = ins; this->buffer_send[2] = p1; this->buffer_send[3] = p2; @@ -508,11 +516,11 @@ namespace hw { } const std::size_t output_index_x = output_index; crypto::public_key derived_pub_x; - hw::ledger::log_hexbuffer("derive_subaddress_public_key: [[IN]] pub ", pub_x.data, 32); - hw::ledger::log_hexbuffer("derive_subaddress_public_key: [[IN]] derivation", derivation_x.data, 32); - hw::ledger::log_message ("derive_subaddress_public_key: [[IN]] index ", std::to_string((int)output_index_x)); + log_hexbuffer("derive_subaddress_public_key: [[IN]] pub ", pub_x.data, 32); + log_hexbuffer("derive_subaddress_public_key: [[IN]] derivation", derivation_x.data, 32); + log_message ("derive_subaddress_public_key: [[IN]] index ", std::to_string((int)output_index_x)); this->controle_device->derive_subaddress_public_key(pub_x, derivation_x,output_index_x,derived_pub_x); - hw::ledger::log_hexbuffer("derive_subaddress_public_key: [[OUT]] derived_pub", derived_pub_x.data, 32); + log_hexbuffer("derive_subaddress_public_key: [[OUT]] derived_pub", derived_pub_x.data, 32); #endif if ((this->mode == TRANSACTION_PARSE) && has_view_key) { @@ -558,11 +566,11 @@ namespace hw { const cryptonote::account_keys keys_x = hw::ledger::decrypt(keys); const cryptonote::subaddress_index index_x = index; crypto::public_key D_x; - hw::ledger::log_hexbuffer("get_subaddress_spend_public_key: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data,32); - hw::ledger::log_hexbuffer("get_subaddress_spend_public_key: [[IN]] keys.m_spend_secret_key", keys_x.m_spend_secret_key.data,32); - hw::ledger::log_message ("get_subaddress_spend_public_key: [[IN]] index ", std::to_string(index_x.major)+"."+std::to_string(index_x.minor)); + log_hexbuffer("get_subaddress_spend_public_key: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data,32); + log_hexbuffer("get_subaddress_spend_public_key: [[IN]] keys.m_spend_secret_key", keys_x.m_spend_secret_key.data,32); + log_message ("get_subaddress_spend_public_key: [[IN]] index ", std::to_string(index_x.major)+"."+std::to_string(index_x.minor)); D_x = this->controle_device->get_subaddress_spend_public_key(keys_x, index_x); - hw::ledger::log_hexbuffer("get_subaddress_spend_public_key: [[OUT]] derivation ", D_x.data, 32); + log_hexbuffer("get_subaddress_spend_public_key: [[OUT]] derivation ", D_x.data, 32); #endif if (index.is_zero()) { @@ -609,14 +617,14 @@ namespace hw { const cryptonote::account_keys keys_x = hw::ledger::decrypt(keys); const cryptonote::subaddress_index index_x = index; cryptonote::account_public_address address_x; - hw::ledger::log_hexbuffer("get_subaddress: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data, 32); - hw::ledger::log_hexbuffer("get_subaddress: [[IN]] keys.m_view_public_key", keys_x.m_account_address.m_view_public_key.data, 32); - hw::ledger::log_hexbuffer("get_subaddress: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data, 32); - hw::ledger::log_hexbuffer("get_subaddress: [[IN]] keys.m_spend_public_key", keys_x.m_account_address.m_spend_public_key.data, 32); - hw::ledger::log_message ("get_subaddress: [[IN]] index ", std::to_string(index_x.major)+"."+std::to_string(index_x.minor)); + log_hexbuffer("get_subaddress: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data, 32); + log_hexbuffer("get_subaddress: [[IN]] keys.m_view_public_key", keys_x.m_account_address.m_view_public_key.data, 32); + log_hexbuffer("get_subaddress: [[IN]] keys.m_view_secret_key ", keys_x.m_view_secret_key.data, 32); + log_hexbuffer("get_subaddress: [[IN]] keys.m_spend_public_key", keys_x.m_account_address.m_spend_public_key.data, 32); + log_message ("get_subaddress: [[IN]] index ", std::to_string(index_x.major)+"."+std::to_string(index_x.minor)); address_x = this->controle_device->get_subaddress(keys_x, index_x); - hw::ledger::log_hexbuffer("get_subaddress: [[OUT]] keys.m_view_public_key ", address_x.m_view_public_key.data, 32); - hw::ledger::log_hexbuffer("get_subaddress: [[OUT]] keys.m_spend_public_key", address_x.m_spend_public_key.data, 32); + log_hexbuffer("get_subaddress: [[OUT]] keys.m_view_public_key ", address_x.m_view_public_key.data, 32); + log_hexbuffer("get_subaddress: [[OUT]] keys.m_spend_public_key", address_x.m_spend_public_key.data, 32); #endif if (index.is_zero()) { @@ -652,10 +660,10 @@ namespace hw { const crypto::secret_key sec_x = hw::ledger::decrypt(sec); const cryptonote::subaddress_index index_x = index; crypto::secret_key sub_sec_x; - hw::ledger::log_message ("get_subaddress_secret_key: [[IN]] index ", std::to_string(index.major)+"."+std::to_string(index.minor)); - hw::ledger::log_hexbuffer("get_subaddress_secret_key: [[IN]] sec ", sec_x.data, 32); + log_message ("get_subaddress_secret_key: [[IN]] index ", std::to_string(index.major)+"."+std::to_string(index.minor)); + log_hexbuffer("get_subaddress_secret_key: [[IN]] sec ", sec_x.data, 32); sub_sec_x = this->controle_device->get_subaddress_secret_key(sec_x, index_x); - hw::ledger::log_hexbuffer("get_subaddress_secret_key: [[OUT]] sub_sec", sub_sec_x.data, 32); + log_hexbuffer("get_subaddress_secret_key: [[OUT]] sub_sec", sub_sec_x.data, 32); #endif int offset = set_command_header_noopt(INS_GET_SUBADDRESS_SECRET_KEY); @@ -717,10 +725,10 @@ namespace hw { const rct::key P_x = P; const rct::key a_x = hw::ledger::decrypt(a); rct::key aP_x; - hw::ledger::log_hexbuffer("scalarmultKey: [[IN]] P ", (char*)P_x.bytes, 32); - hw::ledger::log_hexbuffer("scalarmultKey: [[IN]] a ", (char*)a_x.bytes, 32); + log_hexbuffer("scalarmultKey: [[IN]] P ", (char*)P_x.bytes, 32); + log_hexbuffer("scalarmultKey: [[IN]] a ", (char*)a_x.bytes, 32); this->controle_device->scalarmultKey(aP_x, P_x, a_x); - hw::ledger::log_hexbuffer("scalarmultKey: [[OUT]] aP", (char*)aP_x.bytes, 32); + log_hexbuffer("scalarmultKey: [[OUT]] aP", (char*)aP_x.bytes, 32); #endif int offset = set_command_header_noopt(INS_SECRET_SCAL_MUL_KEY); @@ -752,9 +760,9 @@ namespace hw { #ifdef DEBUG_HWDEVICE const rct::key a_x = hw::ledger::decrypt(a); rct::key aG_x; - hw::ledger::log_hexbuffer("scalarmultKey: [[IN]] a ", (char*)a_x.bytes, 32); + log_hexbuffer("scalarmultKey: [[IN]] a ", (char*)a_x.bytes, 32); this->controle_device->scalarmultBase(aG_x, a_x); - hw::ledger::log_hexbuffer("scalarmultKey: [[OUT]] aG", (char*)aG_x.bytes, 32); + log_hexbuffer("scalarmultKey: [[OUT]] aG", (char*)aG_x.bytes, 32); #endif int offset = set_command_header_noopt(INS_SECRET_SCAL_MUL_BASE); @@ -845,10 +853,10 @@ namespace hw { const crypto::public_key pub_x = pub; const crypto::secret_key sec_x = hw::ledger::decrypt(sec); crypto::key_derivation derivation_x; - hw::ledger::log_hexbuffer("generate_key_derivation: [[IN]] pub ", pub_x.data, 32); - hw::ledger::log_hexbuffer("generate_key_derivation: [[IN]] sec ", sec_x.data, 32); + log_hexbuffer("generate_key_derivation: [[IN]] pub ", pub_x.data, 32); + log_hexbuffer("generate_key_derivation: [[IN]] sec ", sec_x.data, 32); this->controle_device->generate_key_derivation(pub_x, sec_x, derivation_x); - hw::ledger::log_hexbuffer("generate_key_derivation: [[OUT]] derivation", derivation_x.data, 32); + log_hexbuffer("generate_key_derivation: [[OUT]] derivation", derivation_x.data, 32); #endif if ((this->mode == TRANSACTION_PARSE) && has_view_key) { @@ -914,10 +922,10 @@ namespace hw { const crypto::key_derivation derivation_x = hw::ledger::decrypt(derivation); const size_t output_index_x = output_index; crypto::ec_scalar res_x; - hw::ledger::log_hexbuffer("derivation_to_scalar: [[IN]] derivation ", derivation_x.data, 32); - hw::ledger::log_message ("derivation_to_scalar: [[IN]] output_index ", std::to_string(output_index_x)); + log_hexbuffer("derivation_to_scalar: [[IN]] derivation ", derivation_x.data, 32); + log_message ("derivation_to_scalar: [[IN]] output_index ", std::to_string(output_index_x)); this->controle_device->derivation_to_scalar(derivation_x, output_index_x, res_x); - hw::ledger::log_hexbuffer("derivation_to_scalar: [[OUT]] res ", res_x.data, 32); + log_hexbuffer("derivation_to_scalar: [[OUT]] res ", res_x.data, 32); #endif int offset = set_command_header_noopt(INS_DERIVATION_TO_SCALAR); @@ -954,11 +962,11 @@ namespace hw { const std::size_t output_index_x = output_index; const crypto::secret_key sec_x = hw::ledger::decrypt(sec); crypto::secret_key derived_sec_x; - hw::ledger::log_hexbuffer("derive_secret_key: [[IN]] derivation ", derivation_x.data, 32); - hw::ledger::log_message ("derive_secret_key: [[IN]] index ", std::to_string(output_index_x)); - hw::ledger::log_hexbuffer("derive_secret_key: [[IN]] sec ", sec_x.data, 32); + log_hexbuffer("derive_secret_key: [[IN]] derivation ", derivation_x.data, 32); + log_message ("derive_secret_key: [[IN]] index ", std::to_string(output_index_x)); + log_hexbuffer("derive_secret_key: [[IN]] sec ", sec_x.data, 32); this->controle_device->derive_secret_key(derivation_x, output_index_x, sec_x, derived_sec_x); - hw::ledger::log_hexbuffer("derive_secret_key: [[OUT]] derived_sec", derived_sec_x.data, 32); + log_hexbuffer("derive_secret_key: [[OUT]] derived_sec", derived_sec_x.data, 32); #endif int offset = set_command_header_noopt(INS_DERIVE_SECRET_KEY); @@ -998,11 +1006,11 @@ namespace hw { const std::size_t output_index_x = output_index; const crypto::public_key pub_x = pub; crypto::public_key derived_pub_x; - hw::ledger::log_hexbuffer("derive_public_key: [[IN]] derivation ", derivation_x.data, 32); - hw::ledger::log_message ("derive_public_key: [[IN]] output_index", std::to_string(output_index_x)); - hw::ledger::log_hexbuffer("derive_public_key: [[IN]] pub ", pub_x.data, 32); + log_hexbuffer("derive_public_key: [[IN]] derivation ", derivation_x.data, 32); + log_message ("derive_public_key: [[IN]] output_index", std::to_string(output_index_x)); + log_hexbuffer("derive_public_key: [[IN]] pub ", pub_x.data, 32); this->controle_device->derive_public_key(derivation_x, output_index_x, pub_x, derived_pub_x); - hw::ledger::log_hexbuffer("derive_public_key: [[OUT]] derived_pub ", derived_pub_x.data, 32); + log_hexbuffer("derive_public_key: [[OUT]] derived_pub ", derived_pub_x.data, 32); #endif int offset = set_command_header_noopt(INS_DERIVE_PUBLIC_KEY); @@ -1039,11 +1047,11 @@ namespace hw { #ifdef DEBUG_HWDEVICE const crypto::secret_key sec_x = hw::ledger::decrypt(sec); crypto::public_key pub_x; - hw::ledger::log_hexbuffer("secret_key_to_public_key: [[IN]] sec ", sec_x.data, 32); + log_hexbuffer("secret_key_to_public_key: [[IN]] sec ", sec_x.data, 32); bool rc = this->controle_device->secret_key_to_public_key(sec_x, pub_x); - hw::ledger::log_hexbuffer("secret_key_to_public_key: [[OUT]] pub", pub_x.data, 32); + log_hexbuffer("secret_key_to_public_key: [[OUT]] pub", pub_x.data, 32); if (!rc){ - hw::ledger::log_message("secret_key_to_public_key", "secret_key rejected"); + log_message("secret_key_to_public_key", "secret_key rejected"); } #endif @@ -1073,10 +1081,10 @@ namespace hw { const crypto::public_key pub_x = pub; const crypto::secret_key sec_x = hw::ledger::decrypt(sec); crypto::key_image image_x; - hw::ledger::log_hexbuffer("generate_key_image: [[IN]] pub ", pub_x.data, 32); - hw::ledger::log_hexbuffer("generate_key_image: [[IN]] sec ", sec_x.data, 32); + log_hexbuffer("generate_key_image: [[IN]] pub ", pub_x.data, 32); + log_hexbuffer("generate_key_image: [[IN]] sec ", sec_x.data, 32); this->controle_device->generate_key_image(pub_x, sec_x, image_x); - hw::ledger::log_hexbuffer("generate_key_image: [[OUT]] image ", image_x.data, 32); + log_hexbuffer("generate_key_image: [[OUT]] image ", image_x.data, 32); #endif int offset = set_command_header_noopt(INS_GEN_KEY_IMAGE); @@ -1160,10 +1168,139 @@ namespace hw { return true; } - bool device_ledger::add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const size_t real_output_index, - const rct::key &amount_key, const crypto::public_key &out_eph_public_key) { - AUTO_LOCK_CMD(); - key_map.add(ABPkeys(rct::pk2rct(Aout),rct::pk2rct(Bout), is_subaddress, real_output_index, rct::pk2rct(out_eph_public_key), amount_key)); + + bool device_ledger::generate_output_ephemeral_keys(const size_t tx_version, const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, + crypto::public_key &out_eph_public_key) { + AUTO_LOCK_CMD(); + + #ifdef DEBUG_HWDEVICE + const size_t &tx_version_x = tx_version; + const cryptonote::account_keys sender_account_keys_x = sender_account_keys; + memmove((void*)sender_account_keys_x.m_view_secret_key.data, dbg_viewkey.data, 32); + + const crypto::public_key &txkey_pub_x = txkey_pub; + const crypto::secret_key &tx_key_x = tx_key; + const cryptonote::tx_destination_entry &dst_entr_x = dst_entr; + const boost::optional<cryptonote::account_public_address> &change_addr_x = change_addr; + const size_t &output_index_x = output_index; + const bool &need_additional_txkeys_x = need_additional_txkeys; + const std::vector<crypto::secret_key> &additional_tx_keys_x = additional_tx_keys; + std::vector<crypto::public_key> additional_tx_public_keys_x; + std::vector<rct::key> amount_keys_x; + crypto::public_key out_eph_public_key_x; + this->controle_device->generate_output_ephemeral_keys(tx_version_x, sender_account_keys_x, txkey_pub_x, tx_key_x, dst_entr_x, change_addr_x, output_index_x, need_additional_txkeys_x, additional_tx_keys_x, + additional_tx_public_keys_x, amount_keys_x, out_eph_public_key_x); + #endif + + // make additional tx pubkey if necessary + cryptonote::keypair additional_txkey; + if (need_additional_txkeys) { + additional_txkey.sec = additional_tx_keys[output_index]; + } + + //compute derivation, out_eph_public_key, and amount key in one shot on device, to ensure checkable link + const crypto::secret_key *sec; + bool is_change; + + if (change_addr && dst_entr.addr == *change_addr) + { + // sending change to yourself; derivation = a*R + is_change = true; + sec = &sender_account_keys.m_view_secret_key; + } + else + { + is_change = false; + if (dst_entr.is_subaddress && need_additional_txkeys) { + sec = &additional_txkey.sec; + } else { + sec = &tx_key; + } + } + + int offset = set_command_header_noopt(INS_GEN_TXOUT_KEYS); + //tx_version + this->buffer_send[offset+0] = tx_version>>24; + this->buffer_send[offset+1] = tx_version>>16; + this->buffer_send[offset+2] = tx_version>>8; + this->buffer_send[offset+3] = tx_version>>0; + offset += 4; + //tx_sec + memmove(&this->buffer_send[offset], sec->data, 32); + offset += 32; + //Aout + memmove(&this->buffer_send[offset], dst_entr.addr.m_view_public_key.data, 32); + offset += 32; + //Bout + memmove(&this->buffer_send[offset], dst_entr.addr.m_spend_public_key.data, 32); + offset += 32; + //output index + this->buffer_send[offset+0] = output_index>>24; + this->buffer_send[offset+1] = output_index>>16; + this->buffer_send[offset+2] = output_index>>8; + this->buffer_send[offset+3] = output_index>>0; + offset += 4; + //is_change, + this->buffer_send[offset] = is_change; + offset++; + //is_subaddress + this->buffer_send[offset] = dst_entr.is_subaddress; + offset++; + //need_additional_key + this->buffer_send[offset] = need_additional_txkeys; + offset++; + + this->buffer_send[4] = offset-5; + this->length_send = offset; + this->exchange(); + + offset = 0; + unsigned int recv_len = this->length_recv; + if (need_additional_txkeys) + { + ASSERT_X(recv_len>=32, "Not enought data from device"); + memmove(additional_txkey.pub.data, &this->buffer_recv[offset], 32); + additional_tx_public_keys.push_back(additional_txkey.pub); + offset += 32; + recv_len -= 32; + } + if (tx_version > 1) + { + ASSERT_X(recv_len>=32, "Not enought data from device"); + crypto::secret_key scalar1; + memmove(scalar1.data, &this->buffer_recv[offset],32); + amount_keys.push_back(rct::sk2rct(scalar1)); + offset += 32; + recv_len -= 32; + } + ASSERT_X(recv_len>=32, "Not enought data from device"); + memmove(out_eph_public_key.data, &this->buffer_recv[offset], 32); + recv_len -= 32; + + // add ABPkeys + this->add_output_key_mapping(dst_entr.addr.m_view_public_key, dst_entr.addr.m_spend_public_key, dst_entr.is_subaddress, is_change, + need_additional_txkeys, output_index, + amount_keys.back(), out_eph_public_key); + + #ifdef DEBUG_HWDEVICE + hw::ledger::check32("generate_output_ephemeral_keys", "amount_key", (const char*)amount_keys_x.back().bytes, (const char*)hw::ledger::decrypt(amount_keys.back()).bytes); + if (need_additional_txkeys) { + hw::ledger::check32("generate_output_ephemeral_keys", "additional_tx_key", additional_tx_keys_x.back().data, additional_tx_keys.back().data); + } + hw::ledger::check32("generate_output_ephemeral_keys", "out_eph_public_key", out_eph_public_key_x.data, out_eph_public_key.data); + #endif + + return true; + } + + bool device_ledger::add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const bool is_change, + const bool need_additional, const size_t real_output_index, + const rct::key &amount_key, const crypto::public_key &out_eph_public_key) { + key_map.add(ABPkeys(rct::pk2rct(Aout),rct::pk2rct(Bout), is_subaddress, is_change, need_additional, real_output_index, rct::pk2rct(out_eph_public_key), amount_key)); return true; } @@ -1176,7 +1313,10 @@ namespace hw { this->controle_device->ecdhEncode(unmasked_x, AKout_x, short_amount); #endif - int offset = set_command_header_noopt(INS_BLIND); + int offset = set_command_header(INS_BLIND); + //options + this->buffer_send[offset] = short_amount?0x02:0x00; + offset += 1; // AKout memmove(this->buffer_send+offset, AKout.bytes, 32); offset += 32; @@ -1198,7 +1338,7 @@ namespace hw { hw::ledger::check32("ecdhEncode", "amount", (char*)unmasked_x.amount.bytes, (char*)unmasked.amount.bytes); hw::ledger::check32("ecdhEncode", "mask", (char*)unmasked_x.mask.bytes, (char*)unmasked.mask.bytes); - hw::ledger::log_hexbuffer("Blind AKV input", (char*)&this->buffer_recv[64], 3*32); + log_hexbuffer("Blind AKV input", (char*)&this->buffer_recv[64], 3*32); #endif return true; @@ -1213,8 +1353,10 @@ namespace hw { this->controle_device->ecdhDecode(masked_x, AKout_x, short_amount); #endif - int offset = set_command_header_noopt(INS_UNBLIND); - + int offset = set_command_header(INS_UNBLIND); + //options + this->buffer_send[offset] = short_amount?0x02:0x00; + offset += 1; // AKout memmove(this->buffer_send+offset, AKout.bytes, 32); offset += 32; @@ -1310,7 +1452,11 @@ namespace hw { // ====== Aout, Bout, AKout, C, v, k ====== kv_offset = data_offset; - C_offset = kv_offset+ (32*2)*outputs_size; + if (type==rct::RCTTypeBulletproof2) { + C_offset = kv_offset+ (8)*outputs_size; + } else { + C_offset = kv_offset+ (32+32)*outputs_size; + } for ( i = 0; i < outputs_size; i++) { ABPkeys outKeys; bool found; @@ -1323,11 +1469,15 @@ namespace hw { offset = set_command_header(INS_VALIDATE, 0x02, i+1); //options this->buffer_send[offset] = (i==outputs_size-1)? 0x00:0x80 ; + this->buffer_send[offset] |= (type==rct::RCTTypeBulletproof2)?0x02:0x00; offset += 1; if (found) { //is_subaddress this->buffer_send[offset] = outKeys.is_subaddress; offset++; + //is_change_address + this->buffer_send[offset] = outKeys.is_change_address; + offset++; //Aout memmove(this->buffer_send+offset, outKeys.Aout.bytes, 32); offset+=32; @@ -1339,27 +1489,38 @@ namespace hw { offset+=32; } else { // dummy: is_subaddress Aout Bout AKout - offset += 1+32*3; + offset += 2+32*3; } //C memmove(this->buffer_send+offset, data+C_offset,32); offset += 32; C_offset += 32; - //k - memmove(this->buffer_send+offset, data+kv_offset,32); - offset += 32; - //v - kv_offset += 32; - memmove(this->buffer_send+offset, data+kv_offset,32); - offset += 32; - kv_offset += 32; + if (type==rct::RCTTypeBulletproof2) { + //k + memset(this->buffer_send+offset, 0, 32); + offset += 32; + //v + memset(this->buffer_send+offset, 0, 32); + memmove(this->buffer_send+offset, data+kv_offset,8); + offset += 32; + kv_offset += 8; + } else { + //k + memmove(this->buffer_send+offset, data+kv_offset,32); + offset += 32; + kv_offset += 32; + //v + memmove(this->buffer_send+offset, data+kv_offset,32); + offset += 32; + kv_offset += 32; + } this->buffer_send[4] = offset-5; this->length_send = offset; // check transaction user input CHECK_AND_ASSERT_THROW_MES(this->exchange_wait_on_input() == 0, "Transaction denied on device."); #ifdef DEBUG_HWDEVICE - hw::ledger::log_hexbuffer("Prehash AKV input", (char*)&this->buffer_recv[64], 3*32); + log_hexbuffer("Prehash AKV input", (char*)&this->buffer_recv[64], 3*32); #endif } diff --git a/src/device/device_ledger.hpp b/src/device/device_ledger.hpp index 584f1e096..3f470ee7c 100644 --- a/src/device/device_ledger.hpp +++ b/src/device/device_ledger.hpp @@ -56,12 +56,14 @@ namespace hw { rct::key Aout; rct::key Bout; bool is_subaddress; + bool is_change_address; + bool additional_key ; size_t index; rct::key Pout; rct::key AKout; - ABPkeys(const rct::key& A, const rct::key& B, const bool is_subaddr, size_t index, const rct::key& P,const rct::key& AK); + ABPkeys(const rct::key& A, const rct::key& B, const bool is_subaddr, bool is_subaddress, bool is_change_address, size_t index, const rct::key& P,const rct::key& AK); ABPkeys(const ABPkeys& keys) ; - ABPkeys() {index=0;is_subaddress=false;} + ABPkeys() {index=0;is_subaddress=false;is_subaddress=false;is_change_address=false;} }; class Keymap { @@ -105,7 +107,9 @@ namespace hw { device_mode mode; // map public destination key to ephemeral destination key Keymap key_map; - + bool add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const bool is_change, + const bool need_additional, const size_t real_output_index, + const rct::key &amount_key, const crypto::public_key &out_eph_public_key); // To speed up blockchain parsing the view key maybe handle here. crypto::secret_key viewkey; bool has_view_key; @@ -194,9 +198,12 @@ namespace hw { bool ecdhEncode(rct::ecdhTuple & unmasked, const rct::key & sharedSec, bool short_format) override; bool ecdhDecode(rct::ecdhTuple & masked, const rct::key & sharedSec, bool short_format) override; - bool add_output_key_mapping(const crypto::public_key &Aout, const crypto::public_key &Bout, const bool is_subaddress, const size_t real_output_index, - const rct::key &amount_key, const crypto::public_key &out_eph_public_key) override; - + bool generate_output_ephemeral_keys(const size_t tx_version, const cryptonote::account_keys &sender_account_keys, const crypto::public_key &txkey_pub, const crypto::secret_key &tx_key, + const cryptonote::tx_destination_entry &dst_entr, const boost::optional<cryptonote::account_public_address> &change_addr, const size_t output_index, + const bool &need_additional_txkeys, const std::vector<crypto::secret_key> &additional_tx_keys, + std::vector<crypto::public_key> &additional_tx_public_keys, + std::vector<rct::key> &amount_keys, + crypto::public_key &out_eph_public_key) override; bool mlsag_prehash(const std::string &blob, size_t inputs_size, size_t outputs_size, const rct::keyV &hashes, const rct::ctkeyV &outPk, rct::key &prehash) override; bool mlsag_prepare(const rct::key &H, const rct::key &xx, rct::key &a, rct::key &aG, rct::key &aHP, rct::key &rvII) override; diff --git a/src/device/log.cpp b/src/device/log.cpp index c9d3b551b..87505798b 100644 --- a/src/device/log.cpp +++ b/src/device/log.cpp @@ -66,7 +66,7 @@ namespace hw { void decrypt(char* buf, size_t len) { - #ifdef IODUMMYCRYPT_HWDEVICE + #if defined(IODUMMYCRYPT_HWDEVICE) || defined(IONOCRYPT_HWDEVICE) size_t i; if (len == 32) { //view key? @@ -86,11 +86,13 @@ namespace hw { return; } } + #if defined(IODUMMYCRYPT_HWDEVICE) //std decrypt: XOR.55h for (i = 0; i<len;i++) { buf[i] ^= 0x55; } #endif + #endif } crypto::key_derivation decrypt(const crypto::key_derivation &derivation) { diff --git a/src/device_trezor/trezor/transport.hpp b/src/device_trezor/trezor/transport.hpp index 50c31cf73..1cf0daa85 100644 --- a/src/device_trezor/trezor/transport.hpp +++ b/src/device_trezor/trezor/transport.hpp @@ -162,7 +162,7 @@ namespace trezor { m_session(boost::none), m_device_info(boost::none) { - m_http_client.set_server(m_bridge_host, boost::none, false); + m_http_client.set_server(m_bridge_host, boost::none, epee::net_utils::ssl_support_t::e_ssl_support_disabled); } virtual ~BridgeTransport() = default; diff --git a/src/net/CMakeLists.txt b/src/net/CMakeLists.txt new file mode 100644 index 000000000..fdb988f39 --- /dev/null +++ b/src/net/CMakeLists.txt @@ -0,0 +1,34 @@ +# Copyright (c) 2018, The Monero Project +# +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without modification, are +# permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this list of +# conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, this list +# of conditions and the following disclaimer in the documentation and/or other +# materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its contributors may be +# used to endorse or promote products derived from this software without specific +# prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set(net_sources error.cpp parse.cpp socks.cpp tor_address.cpp i2p_address.cpp) +set(net_headers error.h parse.h socks.h tor_address.h i2p_address.h) + +monero_add_library(net ${net_sources} ${net_headers}) +target_link_libraries(net epee ${Boost_ASIO_LIBRARY}) + diff --git a/src/net/error.cpp b/src/net/error.cpp new file mode 100644 index 000000000..037f44d52 --- /dev/null +++ b/src/net/error.cpp @@ -0,0 +1,92 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "error.h" + +#include <string> + +namespace +{ + struct net_category : std::error_category + { + net_category() noexcept + : std::error_category() + {} + + const char* name() const noexcept override + { + return "net::error_category"; + } + + std::string message(int value) const override + { + switch (net::error(value)) + { + case net::error::expected_tld: + return "Expected top-level domain"; + case net::error::invalid_host: + return "Host value is not valid"; + case net::error::invalid_i2p_address: + return "Invalid I2P address"; + case net::error::invalid_port: + return "Invalid port value (expected 0-65535)"; + case net::error::invalid_tor_address: + return "Invalid Tor address"; + case net::error::unsupported_address: + return "Network address not supported"; + default: + break; + } + + return "Unknown net::error"; + } + + std::error_condition default_error_condition(int value) const noexcept override + { + switch (net::error(value)) + { + case net::error::invalid_port: + return std::errc::result_out_of_range; + case net::error::expected_tld: + case net::error::invalid_tor_address: + default: + break; + } + return std::error_condition{value, *this}; + } + }; +} // anonymous + +namespace net +{ + std::error_category const& error_category() noexcept + { + static const net_category instance{}; + return instance; + } +} diff --git a/src/net/error.h b/src/net/error.h new file mode 100644 index 000000000..c8338f7e2 --- /dev/null +++ b/src/net/error.h @@ -0,0 +1,64 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <system_error> +#include <type_traits> + +namespace net +{ + //! General net errors + enum class error : int + { + // 0 reserved for success (as per expect<T>) + expected_tld = 1, //!< Expected a tld + invalid_host, //!< Hostname is not valid + invalid_i2p_address, + invalid_port, //!< Outside of 0-65535 range + invalid_tor_address,//!< Invalid base32 or length + unsupported_address //!< Type not supported by `get_network_address` + }; + + //! \return `std::error_category` for `net` namespace. + std::error_category const& error_category() noexcept; + + //! \return `net::error` as a `std::error_code` value. + inline std::error_code make_error_code(error value) noexcept + { + return std::error_code{int(value), error_category()}; + } +} + +namespace std +{ + template<> + struct is_error_code_enum<::net::error> + : true_type + {}; +} diff --git a/src/net/fwd.h b/src/net/fwd.h new file mode 100644 index 000000000..7cae88251 --- /dev/null +++ b/src/net/fwd.h @@ -0,0 +1,46 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> + +namespace net +{ + enum class error : int; + class tor_address; + class i2p_address; + + namespace socks + { + class client; + template<typename> class connect_handler; + enum class error : int; + enum class version : std::uint8_t; + } +} diff --git a/src/net/i2p_address.cpp b/src/net/i2p_address.cpp new file mode 100644 index 000000000..cba829d3f --- /dev/null +++ b/src/net/i2p_address.cpp @@ -0,0 +1,200 @@ +// Copyright (c) 2019, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "i2p_address.h" + +#include <algorithm> +#include <boost/spirit/include/karma_generate.hpp> +#include <boost/spirit/include/karma_uint.hpp> +#include <cassert> +#include <cstring> +#include <limits> + +#include "net/error.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" +#include "string_tools.h" + +namespace net +{ + namespace + { + // !TODO only b32 addresses right now + constexpr const char tld[] = u8".b32.i2p"; + constexpr const char unknown_host[] = "<unknown i2p host>"; + + constexpr const unsigned b32_length = 52; + + constexpr const char base32_alphabet[] = + u8"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz234567"; + + expect<void> host_check(boost::string_ref host) noexcept + { + if (!host.ends_with(tld)) + return {net::error::expected_tld}; + + host.remove_suffix(sizeof(tld) - 1); + + if (host.size() != b32_length) + return {net::error::invalid_i2p_address}; + if (host.find_first_not_of(base32_alphabet) != boost::string_ref::npos) + return {net::error::invalid_i2p_address}; + + return success(); + } + + struct i2p_serialized + { + std::string host; + std::uint16_t port; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(host) + KV_SERIALIZE(port) + END_KV_SERIALIZE_MAP() + }; + } + + i2p_address::i2p_address(const boost::string_ref host, const std::uint16_t port) noexcept + : port_(port) + { + // this is a private constructor, throw if moved to public + assert(host.size() < sizeof(host_)); + + const std::size_t length = std::min(sizeof(host_) - 1, host.size()); + std::memcpy(host_, host.data(), length); + std::memset(host_ + length, 0, sizeof(host_) - length); + } + + const char* i2p_address::unknown_str() noexcept + { + return unknown_host; + } + + i2p_address::i2p_address() noexcept + : port_(0) + { + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); + std::memset(host_ + sizeof(unknown_host), 0, sizeof(host_) - sizeof(unknown_host)); + } + + expect<i2p_address> i2p_address::make(const boost::string_ref address, const std::uint16_t default_port) + { + boost::string_ref host = address.substr(0, address.rfind(':')); + const boost::string_ref port = + address.substr(host.size() + (host.size() == address.size() ? 0 : 1)); + + MONERO_CHECK(host_check(host)); + + std::uint16_t porti = default_port; + if (!port.empty() && !epee::string_tools::get_xtype_from_string(porti, std::string{port})) + return {net::error::invalid_port}; + + static_assert(b32_length + sizeof(tld) == sizeof(i2p_address::host_), "bad internal host size"); + return i2p_address{host, porti}; + } + + bool i2p_address::_load(epee::serialization::portable_storage& src, epee::serialization::section* hparent) + { + i2p_serialized in{}; + if (in._load(src, hparent) && in.host.size() < sizeof(host_) && (in.host == unknown_host || !host_check(in.host).has_error())) + { + std::memcpy(host_, in.host.data(), in.host.size()); + std::memset(host_ + in.host.size(), 0, sizeof(host_) - in.host.size()); + port_ = in.port; + return true; + } + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); // include null terminator + port_ = 0; + return false; + } + + bool i2p_address::store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const + { + const i2p_serialized out{std::string{host_}, port_}; + return out.store(dest, hparent); + } + + i2p_address::i2p_address(const i2p_address& rhs) noexcept + : port_(rhs.port_) + { + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + + i2p_address& i2p_address::operator=(const i2p_address& rhs) noexcept + { + if (this != std::addressof(rhs)) + { + port_ = rhs.port_; + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + return *this; + } + + bool i2p_address::is_unknown() const noexcept + { + static_assert(1 <= sizeof(host_), "host size too small"); + return host_[0] == '<'; // character is not allowed otherwise + } + + bool i2p_address::equal(const i2p_address& rhs) const noexcept + { + return port_ == rhs.port_ && is_same_host(rhs); + } + + bool i2p_address::less(const i2p_address& rhs) const noexcept + { + return std::strcmp(host_str(), rhs.host_str()) < 0 || port() < rhs.port(); + } + + bool i2p_address::is_same_host(const i2p_address& rhs) const noexcept + { + return std::strcmp(host_str(), rhs.host_str()) == 0; + } + + std::string i2p_address::str() const + { + const std::size_t host_length = std::strlen(host_str()); + const std::size_t port_length = + port_ == 0 ? 0 : std::numeric_limits<std::uint16_t>::digits10 + 2; + + std::string out{}; + out.reserve(host_length + port_length); + out.assign(host_str(), host_length); + + if (port_ != 0) + { + out.push_back(':'); + namespace karma = boost::spirit::karma; + karma::generate(std::back_inserter(out), karma::ushort_, port()); + } + return out; + } +} diff --git a/src/net/i2p_address.h b/src/net/i2p_address.h new file mode 100644 index 000000000..28a1118ba --- /dev/null +++ b/src/net/i2p_address.h @@ -0,0 +1,140 @@ +// Copyright (c) 2019, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> +#include <string> + +#include "common/expect.h" +#include "net/enums.h" +#include "net/error.h" + +namespace epee +{ +namespace serialization +{ + class portable_storage; + struct section; +} +} + +namespace net +{ + //! b32 i2p address; internal format not condensed/decoded. + class i2p_address + { + std::uint16_t port_; + char host_[61]; // null-terminated + + //! Keep in private, `host.size()` has no runtime check + i2p_address(boost::string_ref host, std::uint16_t port) noexcept; + + public: + //! \return Size of internal buffer for host. + static constexpr std::size_t buffer_size() noexcept { return sizeof(host_); } + + //! \return `<unknown tor host>`. + static const char* unknown_str() noexcept; + + //! An object with `port() == 0` and `host_str() == unknown_str()`. + i2p_address() noexcept; + + //! \return A default constructed `i2p_address` object. + static i2p_address unknown() noexcept { return i2p_address{}; } + + /*! + Parse `address` in b32 i2p format (i.e. x.b32.i2p:80) + with `default_port` being used if port is not specified in + `address`. + */ + static expect<i2p_address> make(boost::string_ref address, std::uint16_t default_port = 0); + + //! Load from epee p2p format, and \return false if not valid tor address + bool _load(epee::serialization::portable_storage& src, epee::serialization::section* hparent); + + //! Store in epee p2p format + bool store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const; + + // Moves and copies are currently identical + + i2p_address(const i2p_address& rhs) noexcept; + ~i2p_address() = default; + i2p_address& operator=(const i2p_address& rhs) noexcept; + + //! \return True if default constructed or via `unknown()`. + bool is_unknown() const noexcept; + + bool equal(const i2p_address& rhs) const noexcept; + bool less(const i2p_address& rhs) const noexcept; + + //! \return True if i2p addresses are identical. + bool is_same_host(const i2p_address& rhs) const noexcept; + + //! \return `x.b32.i2p` or `x.b32.i2p:z` if `port() != 0`. + std::string str() const; + + //! \return Null-terminated `x.b32.i2p` value or `unknown_str()`. + const char* host_str() const noexcept { return host_; } + + //! \return Port value or `0` if unspecified. + std::uint16_t port() const noexcept { return port_; } + + static constexpr bool is_loopback() noexcept { return false; } + static constexpr bool is_local() noexcept { return false; } + + static constexpr epee::net_utils::address_type get_type_id() noexcept + { + return epee::net_utils::address_type::i2p; + } + + static constexpr epee::net_utils::zone get_zone() noexcept + { + return epee::net_utils::zone::i2p; + } + + //! \return `!is_unknown()`. + bool is_blockable() const noexcept { return !is_unknown(); } + }; + + inline bool operator==(const i2p_address& lhs, const i2p_address& rhs) noexcept + { + return lhs.equal(rhs); + } + + inline bool operator!=(const i2p_address& lhs, const i2p_address& rhs) noexcept + { + return !lhs.equal(rhs); + } + + inline bool operator<(const i2p_address& lhs, const i2p_address& rhs) noexcept + { + return lhs.less(rhs); + } +} // net diff --git a/src/net/parse.cpp b/src/net/parse.cpp new file mode 100644 index 000000000..eaaadb67e --- /dev/null +++ b/src/net/parse.cpp @@ -0,0 +1,61 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "parse.h" + +#include "net/tor_address.h" +#include "net/i2p_address.h" +#include "string_tools.h" + +namespace net +{ + expect<epee::net_utils::network_address> + get_network_address(const boost::string_ref address, const std::uint16_t default_port) + { + const boost::string_ref host = address.substr(0, address.rfind(':')); + + if (host.empty()) + return make_error_code(net::error::invalid_host); + if (host.ends_with(".onion")) + return tor_address::make(address, default_port); + if (host.ends_with(".i2p")) + return i2p_address::make(address, default_port); + + std::uint16_t port = default_port; + if (host.size() < address.size()) + { + if (!epee::string_tools::get_xtype_from_string(port, std::string{address.substr(host.size() + 1)})) + return make_error_code(net::error::invalid_port); + } + + std::uint32_t ip = 0; + if (epee::string_tools::get_ip_int32_from_string(ip, std::string{host})) + return {epee::net_utils::ipv4_network_address{ip, port}}; + return make_error_code(net::error::unsupported_address); + } +} diff --git a/src/net/parse.h b/src/net/parse.h new file mode 100644 index 000000000..5804c4128 --- /dev/null +++ b/src/net/parse.h @@ -0,0 +1,54 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +#include "common/expect.h" +#include "net/net_utils_base.h" + +namespace net +{ + /*! + Identifies onion, i2p and IPv4 addresses and returns them as a generic + `network_address`. If the type is unsupported, it might be a hostname, + and `error() == net::error::kUnsupportedAddress` is returned. + + \param address An onion address, i2p address, ipv4 address or hostname. Hostname + will return an error. + \param default_port If `address` does not specify a port, this value + will be used. + + \return A tor or IPv4 address, else error. + */ + expect<epee::net_utils::network_address> + get_network_address(boost::string_ref address, std::uint16_t default_port); +} + diff --git a/src/net/socks.cpp b/src/net/socks.cpp new file mode 100644 index 000000000..53154369b --- /dev/null +++ b/src/net/socks.cpp @@ -0,0 +1,316 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "socks.h" + +#include <algorithm> +#include <boost/asio/buffer.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/arithmetic.hpp> +#include <boost/endian/conversion.hpp> +#include <cstring> +#include <limits> +#include <string> + +#include "net/net_utils_base.h" +#include "net/tor_address.h" +#include "net/i2p_address.h" + +namespace net +{ +namespace socks +{ + namespace + { + constexpr const unsigned v4_reply_size = 8; + constexpr const std::uint8_t v4_connect_command = 1; + constexpr const std::uint8_t v4tor_resolve_command = 0xf0; + constexpr const std::uint8_t v4_request_granted = 90; + + struct v4_header + { + std::uint8_t version; + std::uint8_t command_code; + boost::endian::big_uint16_t port; + boost::endian::big_uint32_t ip; + }; + + std::size_t write_domain_header(epee::span<std::uint8_t> out, const std::uint8_t command, const std::uint16_t port, const boost::string_ref domain) + { + if (std::numeric_limits<std::size_t>::max() - sizeof(v4_header) - 2 < domain.size()) + return 0; + + const std::size_t buf_size = sizeof(v4_header) + domain.size() + 2; + if (out.size() < buf_size) + return 0; + + // version 4, 1 indicates invalid ip for domain extension + const v4_header temp{4, command, port, std::uint32_t(1)}; + std::memcpy(out.data(), std::addressof(temp), sizeof(temp)); + out.remove_prefix(sizeof(temp)); + + *(out.data()) = 0; + out.remove_prefix(1); + + std::memcpy(out.data(), domain.data(), domain.size()); + out.remove_prefix(domain.size()); + + *(out.data()) = 0; + return buf_size; + } + + struct socks_category : boost::system::error_category + { + explicit socks_category() noexcept + : boost::system::error_category() + {} + + const char* name() const noexcept override + { + return "net::socks::error_category"; + } + + virtual std::string message(int value) const override + { + switch (socks::error(value)) + { + case socks::error::rejected: + return "Socks request rejected or failed"; + case socks::error::identd_connection: + return "Socks request rejected because server cannot connect to identd on the client"; + case socks::error::identd_user: + return "Socks request rejected because the client program and identd report different user-ids"; + + case socks::error::bad_read: + return "Socks boost::async_read read fewer bytes than expected"; + case socks::error::bad_write: + return "Socks boost::async_write wrote fewer bytes than expected"; + case socks::error::unexpected_version: + return "Socks server returned unexpected version in reply"; + + default: + break; + } + return "Unknown net::socks::error"; + } + + boost::system::error_condition default_error_condition(int value) const noexcept override + { + switch (socks::error(value)) + { + case socks::error::bad_read: + case socks::error::bad_write: + return boost::system::errc::io_error; + case socks::error::unexpected_version: + return boost::system::errc::protocol_error; + default: + break; + }; + if (1 <= value && value <= 256) + return boost::system::errc::protocol_error; + + return boost::system::error_condition{value, *this}; + } + }; + } + + const boost::system::error_category& error_category() noexcept + { + static const socks_category instance{}; + return instance; + } + + struct client::completed + { + std::shared_ptr<client> self_; + + void operator()(const boost::system::error_code error, const std::size_t bytes) const + { + static_assert(1 < sizeof(self_->buffer_), "buffer too small for v4 response"); + + if (self_) + { + client& self = *self_; + self.buffer_size_ = std::min(bytes, sizeof(self.buffer_)); + + if (error) + self.done(error, std::move(self_)); + else if (self.buffer().size() < sizeof(v4_header)) + self.done(socks::error::bad_read, std::move(self_)); + else if (self.buffer_[0] != 0) // response version + self.done(socks::error::unexpected_version, std::move(self_)); + else if (self.buffer_[1] != v4_request_granted) + self.done(socks::error(int(self.buffer_[1]) + 1), std::move(self_)); + else + self.done(boost::system::error_code{}, std::move(self_)); + } + } + }; + + struct client::read + { + std::shared_ptr<client> self_; + + static boost::asio::mutable_buffers_1 get_buffer(client& self) noexcept + { + static_assert(sizeof(v4_header) <= sizeof(self.buffer_), "buffer too small for v4 response"); + return boost::asio::buffer(self.buffer_, sizeof(v4_header)); + } + + void operator()(const boost::system::error_code error, const std::size_t bytes) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else if (bytes < self.buffer().size()) + self.done(socks::error::bad_write, std::move(self_)); + else + boost::asio::async_read(self.proxy_, get_buffer(self), completed{std::move(self_)}); + } + } + }; + + struct client::write + { + std::shared_ptr<client> self_; + + static boost::asio::const_buffers_1 get_buffer(client const& self) noexcept + { + return boost::asio::buffer(self.buffer_, self.buffer_size_); + } + + void operator()(const boost::system::error_code error) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else + boost::asio::async_write(self.proxy_, get_buffer(self), read{std::move(self_)}); + } + } + }; + + client::client(stream_type::socket&& proxy, socks::version ver) + : proxy_(std::move(proxy)), buffer_size_(0), buffer_(), ver_(ver) + {} + + client::~client() {} + + bool client::set_connect_command(const epee::net_utils::ipv4_network_address& address) + { + switch (socks_version()) + { + case version::v4: + case version::v4a: + case version::v4a_tor: + break; + default: + return false; + } + + static_assert(sizeof(v4_header) < sizeof(buffer_), "buffer size too small for request"); + static_assert(0 < sizeof(buffer_), "buffer size too small for null termination"); + + // version 4 + const v4_header temp{4, v4_connect_command, address.port(), boost::endian::big_to_native(address.ip())}; + std::memcpy(std::addressof(buffer_), std::addressof(temp), sizeof(temp)); + buffer_[sizeof(temp)] = 0; + buffer_size_ = sizeof(temp) + 1; + + return true; + } + + bool client::set_connect_command(const boost::string_ref domain, std::uint16_t port) + { + switch (socks_version()) + { + case version::v4a: + case version::v4a_tor: + break; + + default: + return false; + } + + const std::size_t buf_used = write_domain_header(buffer_, v4_connect_command, port, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::set_connect_command(const net::tor_address& address) + { + if (!address.is_unknown()) + return set_connect_command(address.host_str(), address.port()); + return false; + } + + bool client::set_connect_command(const net::i2p_address& address) + { + if (!address.is_unknown()) + return set_connect_command(address.host_str(), address.port()); + return false; + } + + bool client::set_resolve_command(boost::string_ref domain) + { + if (socks_version() != version::v4a_tor) + return false; + + const std::size_t buf_used = write_domain_header(buffer_, v4tor_resolve_command, 0, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + alias.proxy_.async_connect(proxy_address, write{std::move(self)}); + return true; + } + return false; + } + + bool client::send(std::shared_ptr<client> self) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + boost::asio::async_write(alias.proxy_, write::get_buffer(alias), read{std::move(self)}); + return true; + } + return false; + } +} // socks +} // net diff --git a/src/net/socks.h b/src/net/socks.h new file mode 100644 index 000000000..825937792 --- /dev/null +++ b/src/net/socks.h @@ -0,0 +1,228 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/system/error_code.hpp> +#include <boost/type_traits/integral_constant.hpp> +#include <boost/utility/string_ref.hpp> +#include <memory> +#include <utility> + +#include "net/fwd.h" +#include "span.h" + +namespace epee +{ +namespace net_utils +{ + class ipv4_network_address; +} +} + +namespace net +{ +namespace socks +{ + //! Supported socks variants. + enum class version : std::uint8_t + { + v4 = 0, + v4a, + v4a_tor //!< Extensions defined in Tor codebase + }; + + //! Possible errors with socks communication. Defined in https://www.openssh.com/txt/socks4.protocol + enum class error : int + { + // 0 is reserved for success value + // 1-256 -> reserved for error values from socks server (+1 from wire value). + rejected = 92, + identd_connection, + identd_user, + // Specific to application + bad_read = 257, + bad_write, + unexpected_version + }; + + /* boost::system::error_code is extended for easier compatibility with + boost::asio errors. If std::error_code is needed (with expect<T> for + instance), then upgrade to boost 1.65+ or use conversion code in + develop branch at boost/system/detail/std_interoperability.hpp */ + + //! \return boost::system::error_category for net::socks namespace + const boost::system::error_category& error_category() noexcept; + + //! \return net::socks::error as a boost::system::error_code. + inline boost::system::error_code make_error_code(error value) noexcept + { + return boost::system::error_code{int(value), socks::error_category()}; + } + + //! Client support for socks connect and resolve commands. + class client + { + boost::asio::ip::tcp::socket proxy_; + std::uint16_t buffer_size_; + std::uint8_t buffer_[1024]; + socks::version ver_; + + /*! + Only invoked after `*send(...)` function completes or fails. + `bool(error) == false` indicates success; `self.get()` is always + `this` and allows implementations to skip + `std::enable_shared_from_this<T>` (ASIO callbacks need shared_ptr). + The design saves space and reduces cycles (everything uses moves, + so no atomic operations are ever necessary). + + \param error when processing last command (if any). + \param self `shared_ptr<client>` handle to `this`. + */ + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) = 0; + + public: + using stream_type = boost::asio::ip::tcp; + + // defined in cpp + struct write; + struct read; + struct completed; + + /*! + \param proxy ownership is passed into `this`. Does not have to be + in connected state. + \param ver socks version for the connection. + */ + explicit client(stream_type::socket&& proxy, socks::version ver); + + client(const client&) = delete; + virtual ~client(); + client& operator=(const client&) = delete; + + //! \return Ownership of socks client socket object. + stream_type::socket take_socket() + { + return stream_type::socket{std::move(proxy_)}; + } + + //! \return Socks version. + socks::version socks_version() const noexcept { return ver_; } + + //! \return Contents of internal buffer. + epee::span<const std::uint8_t> buffer() const noexcept + { + return {buffer_, buffer_size_}; + } + + //! \post `buffer.empty()`. + void clear_command() noexcept { buffer_size_ = 0; } + + //! Try to set `address` as remote connection request. + bool set_connect_command(const epee::net_utils::ipv4_network_address& address); + + //! Try to set `domain` + `port` as remote connection request. + bool set_connect_command(boost::string_ref domain, std::uint16_t port); + + //! Try to set `address` as remote Tor hidden service connection request. + bool set_connect_command(const net::tor_address& address); + + //! Try to set `address` as remote i2p hidden service connection request. + bool set_connect_command(const net::i2p_address& address); + + //! Try to set `domain` as remote DNS A record lookup request. + bool set_resolve_command(boost::string_ref domain); + + /*! + Asynchronously connect to `proxy_address` then issue command in + `buffer()`. The `done(...)` method will be invoked upon completion + with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + this function. + + \param self ownership of object is given to function. + \param proxy_address of the socks server. + \return False if `self->buffer().empty()` (no command set). + */ + static bool connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address); + + /*! + Assume existing connection to proxy server; asynchronously issue + command in `buffer()`. The `done(...)` method will be invoked + upon completion with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + the function. + + \param self ownership of object is given to function. + \return False if `self->buffer().empty()` (no command set). + */ + static bool send(std::shared_ptr<client> self); + }; + + template<typename Handler> + class connect_client : public client + { + Handler handler_; + + virtual void done(boost::system::error_code error, std::shared_ptr<client>) override + { + handler_(error, take_socket()); + } + + public: + explicit connect_client(stream_type::socket&& proxy, socks::version ver, Handler&& handler) + : client(std::move(proxy), ver), handler_(std::move(handler)) + {} + + virtual ~connect_client() override {} + }; + + template<typename Handler> + inline std::shared_ptr<client> + make_connect_client(client::stream_type::socket&& proxy, socks::version ver, Handler handler) + { + return std::make_shared<connect_client<Handler>>(std::move(proxy), ver, std::move(handler)); + } +} // socks +} // net + +namespace boost +{ +namespace system +{ + template<> + struct is_error_code_enum<net::socks::error> + : true_type + {}; +} // system +} // boost diff --git a/src/net/tor_address.cpp b/src/net/tor_address.cpp new file mode 100644 index 000000000..904a9a0fc --- /dev/null +++ b/src/net/tor_address.cpp @@ -0,0 +1,203 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "tor_address.h" + +#include <algorithm> +#include <boost/spirit/include/karma_generate.hpp> +#include <boost/spirit/include/karma_uint.hpp> +#include <cassert> +#include <cstring> +#include <limits> + +#include "net/error.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" +#include "string_tools.h" + +namespace net +{ + namespace + { + constexpr const char tld[] = u8".onion"; + constexpr const char unknown_host[] = "<unknown tor host>"; + + constexpr const unsigned v2_length = 16; + constexpr const unsigned v3_length = 56; + + constexpr const char base32_alphabet[] = + u8"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz234567"; + + expect<void> host_check(boost::string_ref host) noexcept + { + if (!host.ends_with(tld)) + return {net::error::expected_tld}; + + host.remove_suffix(sizeof(tld) - 1); + + //! \TODO v3 has checksum, base32 decoding is required to verify it + if (host.size() != v2_length && host.size() != v3_length) + return {net::error::invalid_tor_address}; + if (host.find_first_not_of(base32_alphabet) != boost::string_ref::npos) + return {net::error::invalid_tor_address}; + + return success(); + } + + struct tor_serialized + { + std::string host; + std::uint16_t port; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(host) + KV_SERIALIZE(port) + END_KV_SERIALIZE_MAP() + }; + } + + tor_address::tor_address(const boost::string_ref host, const std::uint16_t port) noexcept + : port_(port) + { + // this is a private constructor, throw if moved to public + assert(host.size() < sizeof(host_)); + + const std::size_t length = std::min(sizeof(host_) - 1, host.size()); + std::memcpy(host_, host.data(), length); + std::memset(host_ + length, 0, sizeof(host_) - length); + } + + const char* tor_address::unknown_str() noexcept + { + return unknown_host; + } + + tor_address::tor_address() noexcept + : port_(0) + { + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); + std::memset(host_ + sizeof(unknown_host), 0, sizeof(host_) - sizeof(unknown_host)); + } + + expect<tor_address> tor_address::make(const boost::string_ref address, const std::uint16_t default_port) + { + boost::string_ref host = address.substr(0, address.rfind(':')); + const boost::string_ref port = + address.substr(host.size() + (host.size() == address.size() ? 0 : 1)); + + MONERO_CHECK(host_check(host)); + + std::uint16_t porti = default_port; + if (!port.empty() && !epee::string_tools::get_xtype_from_string(porti, std::string{port})) + return {net::error::invalid_port}; + + static_assert(v2_length <= v3_length, "bad internal host size"); + static_assert(v3_length + sizeof(tld) == sizeof(tor_address::host_), "bad internal host size"); + return tor_address{host, porti}; + } + + bool tor_address::_load(epee::serialization::portable_storage& src, epee::serialization::section* hparent) + { + tor_serialized in{}; + if (in._load(src, hparent) && in.host.size() < sizeof(host_) && (in.host == unknown_host || !host_check(in.host).has_error())) + { + std::memcpy(host_, in.host.data(), in.host.size()); + std::memset(host_ + in.host.size(), 0, sizeof(host_) - in.host.size()); + port_ = in.port; + return true; + } + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); // include null terminator + port_ = 0; + return false; + } + + bool tor_address::store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const + { + const tor_serialized out{std::string{host_}, port_}; + return out.store(dest, hparent); + } + + tor_address::tor_address(const tor_address& rhs) noexcept + : port_(rhs.port_) + { + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + + tor_address& tor_address::operator=(const tor_address& rhs) noexcept + { + if (this != std::addressof(rhs)) + { + port_ = rhs.port_; + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + return *this; + } + + bool tor_address::is_unknown() const noexcept + { + static_assert(1 <= sizeof(host_), "host size too small"); + return host_[0] == '<'; // character is not allowed otherwise + } + + bool tor_address::equal(const tor_address& rhs) const noexcept + { + return port_ == rhs.port_ && is_same_host(rhs); + } + + bool tor_address::less(const tor_address& rhs) const noexcept + { + return std::strcmp(host_str(), rhs.host_str()) < 0 || port() < rhs.port(); + } + + bool tor_address::is_same_host(const tor_address& rhs) const noexcept + { + //! \TODO v2 and v3 should be comparable - requires base32 + return std::strcmp(host_str(), rhs.host_str()) == 0; + } + + std::string tor_address::str() const + { + const std::size_t host_length = std::strlen(host_str()); + const std::size_t port_length = + port_ == 0 ? 0 : std::numeric_limits<std::uint16_t>::digits10 + 2; + + std::string out{}; + out.reserve(host_length + port_length); + out.assign(host_str(), host_length); + + if (port_ != 0) + { + out.push_back(':'); + namespace karma = boost::spirit::karma; + karma::generate(std::back_inserter(out), karma::ushort_, port()); + } + return out; + } +} diff --git a/src/net/tor_address.h b/src/net/tor_address.h new file mode 100644 index 000000000..22d8cc119 --- /dev/null +++ b/src/net/tor_address.h @@ -0,0 +1,140 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> +#include <string> + +#include "common/expect.h" +#include "net/enums.h" +#include "net/error.h" + +namespace epee +{ +namespace serialization +{ + class portable_storage; + struct section; +} +} + +namespace net +{ + //! Tor onion address; internal format not condensed/decoded. + class tor_address + { + std::uint16_t port_; + char host_[63]; // null-terminated + + //! Keep in private, `host.size()` has no runtime check + tor_address(boost::string_ref host, std::uint16_t port) noexcept; + + public: + //! \return Size of internal buffer for host. + static constexpr std::size_t buffer_size() noexcept { return sizeof(host_); } + + //! \return `<unknown tor host>`. + static const char* unknown_str() noexcept; + + //! An object with `port() == 0` and `host_str() == unknown_str()`. + tor_address() noexcept; + + //! \return A default constructed `tor_address` object. + static tor_address unknown() noexcept { return tor_address{}; } + + /*! + Parse `address` in onion v2 or v3 format with (i.e. x.onion:80) + with `default_port` being used iff port is not specified in + `address`. + */ + static expect<tor_address> make(boost::string_ref address, std::uint16_t default_port = 0); + + //! Load from epee p2p format, and \return false if not valid tor address + bool _load(epee::serialization::portable_storage& src, epee::serialization::section* hparent); + + //! Store in epee p2p format + bool store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const; + + // Moves and copies are currently identical + + tor_address(const tor_address& rhs) noexcept; + ~tor_address() = default; + tor_address& operator=(const tor_address& rhs) noexcept; + + //! \return True if default constructed or via `unknown()`. + bool is_unknown() const noexcept; + + bool equal(const tor_address& rhs) const noexcept; + bool less(const tor_address& rhs) const noexcept; + + //! \return True if onion addresses are identical. + bool is_same_host(const tor_address& rhs) const noexcept; + + //! \return `x.onion` or `x.onion:z` if `port() != 0`. + std::string str() const; + + //! \return Null-terminated `x.onion` value or `unknown_str()`. + const char* host_str() const noexcept { return host_; } + + //! \return Port value or `0` if unspecified. + std::uint16_t port() const noexcept { return port_; } + + static constexpr bool is_loopback() noexcept { return false; } + static constexpr bool is_local() noexcept { return false; } + + static constexpr epee::net_utils::address_type get_type_id() noexcept + { + return epee::net_utils::address_type::tor; + } + + static constexpr epee::net_utils::zone get_zone() noexcept + { + return epee::net_utils::zone::tor; + } + + //! \return `!is_unknown()`. + bool is_blockable() const noexcept { return !is_unknown(); } + }; + + inline bool operator==(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.equal(rhs); + } + + inline bool operator!=(const tor_address& lhs, const tor_address& rhs) noexcept + { + return !lhs.equal(rhs); + } + + inline bool operator<(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.less(rhs); + } +} // net diff --git a/src/p2p/CMakeLists.txt b/src/p2p/CMakeLists.txt index 9b924907e..9a1730b8a 100644 --- a/src/p2p/CMakeLists.txt +++ b/src/p2p/CMakeLists.txt @@ -40,6 +40,7 @@ target_link_libraries(p2p PUBLIC version cryptonote_core + net ${UPNP_LIBRARIES} ${Boost_CHRONO_LIBRARY} ${Boost_PROGRAM_OPTIONS_LIBRARY} diff --git a/src/p2p/net_node.cpp b/src/p2p/net_node.cpp index 7cad6e077..2f0678913 100644 --- a/src/p2p/net_node.cpp +++ b/src/p2p/net_node.cpp @@ -28,9 +28,83 @@ // // Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers +#include <boost/algorithm/string/find_iterator.hpp> +#include <boost/algorithm/string/finder.hpp> +#include <boost/chrono/duration.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/optional/optional.hpp> +#include <boost/thread/future.hpp> +#include <boost/utility/string_ref.hpp> +#include <chrono> +#include <utility> + #include "common/command_line.h" #include "cryptonote_core/cryptonote_core.h" +#include "cryptonote_protocol/cryptonote_protocol_defs.h" #include "net_node.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "net/i2p_address.h" +#include "p2p/p2p_protocol_defs.h" +#include "string_tools.h" + +namespace +{ + constexpr const boost::chrono::milliseconds future_poll_interval{500}; + constexpr const std::chrono::seconds socks_connect_timeout{P2P_DEFAULT_SOCKS_CONNECT_TIMEOUT}; + + std::int64_t get_max_connections(const boost::iterator_range<boost::string_ref::const_iterator> value) noexcept + { + // -1 is default, 0 is error + if (value.empty()) + return -1; + + std::uint32_t out = 0; + if (epee::string_tools::get_xtype_from_string(out, std::string{value.begin(), value.end()})) + return out; + return 0; + } + + template<typename T> + epee::net_utils::network_address get_address(const boost::string_ref value) + { + expect<T> address = T::make(value); + if (!address) + { + MERROR( + "Failed to parse " << epee::net_utils::zone_to_string(T::get_zone()) << " address \"" << value << "\": " << address.error().message() + ); + return {}; + } + return {std::move(*address)}; + } + + bool start_socks(std::shared_ptr<net::socks::client> client, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + CHECK_AND_ASSERT_MES(client != nullptr, false, "Unexpected null client"); + + bool set = false; + switch (remote.get_type_id()) + { + case net::tor_address::get_type_id(): + set = client->set_connect_command(remote.as<net::tor_address>()); + break; + case net::i2p_address::get_type_id(): + set = client->set_connect_command(remote.as<net::i2p_address>()); + break; + default: + MERROR("Unsupported network address in socks_connect"); + return false; + } + + const bool sent = + set && net::socks::client::connect_and_send(std::move(client), proxy); + CHECK_AND_ASSERT_MES(sent, false, "Unexpected failure to init socks client"); + return true; + } +} namespace nodetool { @@ -55,6 +129,8 @@ namespace nodetool const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node = {"add-exclusive-node", "Specify list of peers to connect to only." " If this option is given the options add-priority-node and seed-node are ignored"}; const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node = {"seed-node", "Connect to a node to retrieve peer addresses, and disconnect"}; + const command_line::arg_descriptor<std::vector<std::string> > arg_proxy = {"proxy", "<network-type>,<socks-ip:port>[,max_connections] i.e. \"tor,127.0.0.1:9050,100\""}; + const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound = {"anonymous-inbound", "<hidden-service-address>,<[bind-ip:]port>[,max_connections] i.e. \"x.onion,127.0.0.1:18083,100\""}; const command_line::arg_descriptor<bool> arg_p2p_hide_my_port = {"hide-my-port", "Do not announce yourself as peerlist candidate", false, true}; const command_line::arg_descriptor<bool> arg_no_igd = {"no-igd", "Disable UPnP port mapping"}; @@ -67,4 +143,203 @@ namespace nodetool const command_line::arg_descriptor<int64_t> arg_limit_rate = {"limit-rate", "set limit-rate [kB/s]", -1}; const command_line::arg_descriptor<bool> arg_save_graph = {"save-graph", "Save data for dr monero", false}; + + boost::optional<std::vector<proxy>> get_proxies(boost::program_options::variables_map const& vm) + { + namespace ip = boost::asio::ip; + + std::vector<proxy> proxies{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_proxy); + proxies.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + proxies.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No network type for --" << arg_proxy.name); + const boost::string_ref zone{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No ipv4:port given for --" << arg_proxy.name); + const boost::string_ref proxy{next->begin(), next->size()}; + + ++next; + if (!next.eof()) + { + proxies.back().max_connections = get_max_connections(*next); + if (proxies.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + switch (epee::net_utils::zone_from_string(zone)) + { + case epee::net_utils::zone::tor: + proxies.back().zone = epee::net_utils::zone::tor; + break; + case epee::net_utils::zone::i2p: + proxies.back().zone = epee::net_utils::zone::i2p; + break; + default: + MERROR("Invalid network for --" << arg_proxy.name); + return boost::none; + } + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{proxy}) || port == 0) + { + MERROR("Invalid ipv4:port given for --" << arg_proxy.name); + return boost::none; + } + proxies.back().address = ip::tcp::endpoint{ip::address_v4{boost::endian::native_to_big(ip)}, port}; + } + + return proxies; + } + + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(boost::program_options::variables_map const& vm) + { + std::vector<anonymous_inbound> inbounds{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_anonymous_inbound); + inbounds.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + inbounds.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No inbound address for --" << arg_anonymous_inbound.name); + const boost::string_ref address{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No local ipv4:port given for --" << arg_anonymous_inbound.name); + const boost::string_ref bind{next->begin(), next->size()}; + + const std::size_t colon = bind.find_first_of(':'); + CHECK_AND_ASSERT_MES(colon < bind.size(), boost::none, "No local port given for --" << arg_anonymous_inbound.name); + + ++next; + if (!next.eof()) + { + inbounds.back().max_connections = get_max_connections(*next); + if (inbounds.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + expect<epee::net_utils::network_address> our_address = net::get_network_address(address, 0); + switch (our_address ? our_address->get_type_id() : epee::net_utils::address_type::invalid) + { + case net::tor_address::get_type_id(): + inbounds.back().our_address = std::move(*our_address); + inbounds.back().default_remote = net::tor_address::unknown(); + break; + case net::i2p_address::get_type_id(): + inbounds.back().our_address = std::move(*our_address); + inbounds.back().default_remote = net::i2p_address::unknown(); + break; + default: + MERROR("Invalid inbound address (" << address << ") for --" << arg_anonymous_inbound.name << ": " << (our_address ? "invalid type" : our_address.error().message())); + return boost::none; + } + + // get_address returns default constructed address on error + if (inbounds.back().our_address == epee::net_utils::network_address{}) + return boost::none; + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{bind})) + { + MERROR("Invalid ipv4:port given for --" << arg_anonymous_inbound.name); + return boost::none; + } + inbounds.back().local_ip = std::string{bind.substr(0, colon)}; + inbounds.back().local_port = std::string{bind.substr(colon + 1)}; + } + + return inbounds; + } + + bool is_filtered_command(const epee::net_utils::network_address& address, int command) + { + switch (command) + { + case nodetool::COMMAND_HANDSHAKE_T<cryptonote::CORE_SYNC_DATA>::ID: + case nodetool::COMMAND_TIMED_SYNC_T<cryptonote::CORE_SYNC_DATA>::ID: + case cryptonote::NOTIFY_NEW_TRANSACTIONS::ID: + return false; + default: + break; + } + + if (address.get_zone() == epee::net_utils::zone::public_) + return false; + + MWARNING("Filtered command (#" << command << ") to/from " << address.str()); + return true; + } + + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + using socket_type = net::socks::client::stream_type::socket; + using client_result = std::pair<boost::system::error_code, socket_type>; + + struct notify + { + boost::promise<client_result> socks_promise; + + void operator()(boost::system::error_code error, socket_type&& sock) + { + socks_promise.set_value(std::make_pair(error, std::move(sock))); + } + }; + + boost::unique_future<client_result> socks_result{}; + { + boost::promise<client_result> socks_promise{}; + socks_result = socks_promise.get_future(); + + auto client = net::socks::make_connect_client( + boost::asio::ip::tcp::socket{service}, net::socks::version::v4a, notify{std::move(socks_promise)} + ); + if (!start_socks(std::move(client), proxy, remote)) + return boost::none; + } + + const auto start = std::chrono::steady_clock::now(); + while (socks_result.wait_for(future_poll_interval) == boost::future_status::timeout) + { + if (socks_connect_timeout < std::chrono::steady_clock::now() - start) + { + MERROR("Timeout on socks connect (" << proxy << " to " << remote.str() << ")"); + return boost::none; + } + + if (stop_signal) + return boost::none; + } + + try + { + auto result = socks_result.get(); + if (!result.first) + return {std::move(result.second)}; + + MERROR("Failed to make socks connection to " << remote.str() << " (via " << proxy << "): " << result.first.message()); + } + catch (boost::broken_promise const&) + {} + + return boost::none; + } } diff --git a/src/p2p/net_node.h b/src/p2p/net_node.h index 09c219a44..58e3c8857 100644 --- a/src/p2p/net_node.h +++ b/src/p2p/net_node.h @@ -30,12 +30,17 @@ #pragma once #include <array> +#include <atomic> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> #include <boost/thread.hpp> +#include <boost/optional/optional_fwd.hpp> #include <boost/program_options/options_description.hpp> #include <boost/program_options/variables_map.hpp> -#include <boost/serialization/version.hpp> #include <boost/uuid/uuid.hpp> -#include <boost/serialization/map.hpp> +#include <functional> +#include <utility> +#include <vector> #include "cryptonote_config.h" #include "warnings.h" @@ -47,6 +52,8 @@ #include "net_peerlist.h" #include "math_helper.h" #include "net_node_common.h" +#include "net/enums.h" +#include "net/fwd.h" #include "common/command_line.h" PUSH_WARNINGS @@ -54,6 +61,47 @@ DISABLE_VS_WARNINGS(4355) namespace nodetool { + struct proxy + { + proxy() + : max_connections(-1), + address(), + zone(epee::net_utils::zone::invalid) + {} + + std::int64_t max_connections; + boost::asio::ip::tcp::endpoint address; + epee::net_utils::zone zone; + }; + + struct anonymous_inbound + { + anonymous_inbound() + : max_connections(-1), + local_ip(), + local_port(), + our_address(), + default_remote() + {} + + std::int64_t max_connections; + std::string local_ip; + std::string local_port; + epee::net_utils::network_address our_address; + epee::net_utils::network_address default_remote; + }; + + boost::optional<std::vector<proxy>> get_proxies(const boost::program_options::variables_map& vm); + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(const boost::program_options::variables_map& vm); + + //! \return True if `commnd` is filtered (ignored/dropped) for `address` + bool is_filtered_command(epee::net_utils::network_address const& address, int command); + + // hides boost::future and chrono stuff from mondo template file + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote); + + template<class base_type> struct p2p_connection_context_t: base_type //t_payload_net_handler::connection_context //public net_utils::connection_context_base { @@ -78,53 +126,124 @@ namespace nodetool typedef COMMAND_HANDSHAKE_T<typename t_payload_net_handler::payload_type> COMMAND_HANDSHAKE; typedef COMMAND_TIMED_SYNC_T<typename t_payload_net_handler::payload_type> COMMAND_TIMED_SYNC; + typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context>> net_server; + + struct network_zone; + using connect_func = boost::optional<p2p_connection_context>(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t); + + struct config + { + config() + : m_net_config(), + m_peer_id(crypto::rand<uint64_t>()), + m_support_flags(0) + {} + + network_config m_net_config; + uint64_t m_peer_id; + uint32_t m_support_flags; + }; + + struct network_zone + { + network_zone() + : m_connect(nullptr), + m_net_server(epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + network_zone(boost::asio::io_service& public_service) + : m_connect(nullptr), + m_net_server(public_service, epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + connect_func* m_connect; + net_server m_net_server; + std::string m_bind_ip; + std::string m_port; + epee::net_utils::network_address m_our_address; // in anonymity networks + peerlist_manager m_peerlist; + config m_config; + boost::asio::ip::tcp::endpoint m_proxy_address; + std::atomic<unsigned int> m_current_number_of_out_peers; + std::atomic<unsigned int> m_current_number_of_in_peers; + bool m_can_pingback; + + private: + void set_config_defaults() noexcept + { + // at this moment we have a hardcoded config + m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; + m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; + m_config.m_net_config.config_id = 0; + m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; + m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; + m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; + m_config.m_support_flags = 0; // only set in public zone + } + }; + public: typedef t_payload_net_handler payload_net_handler; node_server(t_payload_net_handler& payload_handler) - :m_payload_handler(payload_handler), - m_current_number_of_out_peers(0), - m_current_number_of_in_peers(0), - m_allow_local_ip(false), - m_hide_my_port(false), - m_no_igd(false), - m_offline(false), - m_save_graph(false), - is_closing(false), - m_net_server( epee::net_utils::e_connection_type_P2P ) // this is a P2P connection of the main p2p node server, because this is class node_server<> - {} - virtual ~node_server() + : m_payload_handler(payload_handler), + m_external_port(0), + m_allow_local_ip(false), + m_hide_my_port(false), + m_no_igd(false), + m_offline(false), + m_save_graph(false), + is_closing(false), + m_network_id() {} + virtual ~node_server(); static void init_options(boost::program_options::options_description& desc); bool run(); + network_zone& add_zone(epee::net_utils::zone zone); bool init(const boost::program_options::variables_map& vm); bool deinit(); bool send_stop_signal(); uint32_t get_this_peer_port(){return m_listening_port;} t_payload_net_handler& get_payload_object(); - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - a & m_peerlist; - if (ver == 0) - { - // from v1, we do not store the peer id anymore - peerid_type peer_id = AUTO_VAL_INIT (peer_id); - a & peer_id; - } - } // debug functions bool log_peerlist(); bool log_connections(); - virtual uint64_t get_connections_count(); - size_t get_outgoing_connections_count(); - size_t get_incoming_connections_count(); - peerlist_manager& get_peerlist_manager(){return m_peerlist;} - void delete_out_connections(size_t count); - void delete_in_connections(size_t count); + + // These functions only return information for the "public" zone + virtual uint64_t get_public_connections_count(); + size_t get_public_outgoing_connections_count(); + size_t get_public_white_peers_count(); + size_t get_public_gray_peers_count(); + void get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white); + size_t get_zone_count() const { return m_network_zones.size(); } + + void change_max_out_public_peers(size_t count); + void change_max_in_public_peers(size_t count); virtual bool block_host(const epee::net_utils::network_address &adress, time_t seconds = P2P_IP_BLOCKTIME); virtual bool unblock_host(const epee::net_utils::network_address &address); virtual std::map<std::string, time_t> get_blocked_hosts() { CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); return m_blocked_hosts; } @@ -150,6 +269,9 @@ namespace nodetool CHAIN_LEVIN_NOTIFY_MAP2(p2p_connection_context); //move levin_commands_handler interface notify(...) callbacks into nothing BEGIN_INVOKE_MAP2(node_server) + if (is_filtered_command(context.m_remote_address, command)) + return LEVIN_ERROR_CONNECTION_HANDLER_NOT_DEFINED; + HANDLE_INVOKE_T2(COMMAND_HANDSHAKE, &node_server::handle_handshake) HANDLE_INVOKE_T2(COMMAND_TIMED_SYNC, &node_server::handle_timed_sync) HANDLE_INVOKE_T2(COMMAND_PING, &node_server::handle_ping) @@ -178,7 +300,7 @@ namespace nodetool bool make_default_peer_id(); bool make_default_config(); bool store_config(); - bool check_trust(const proof_of_trust& tr); + bool check_trust(const proof_of_trust& tr, epee::net_utils::zone zone_type); //----------------- levin_commands_handler ------------------------------------------------------------- @@ -186,8 +308,7 @@ namespace nodetool virtual void on_connection_close(p2p_connection_context& context); virtual void callback(p2p_connection_context& context); //----------------- i_p2p_endpoint ------------------------------------------------------------- - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections); - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context); + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections); virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context); virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context); virtual bool drop_connection(const epee::net_utils::connection_context_base& context); @@ -204,7 +325,7 @@ namespace nodetool ); bool idle_worker(); bool handle_remote_peerlist(const std::vector<peerlist_entry>& peerlist, time_t local_time, const epee::net_utils::connection_context_base& context); - bool get_local_node_data(basic_node_data& node_data); + bool get_local_node_data(basic_node_data& node_data, const network_zone& zone); //bool get_local_handshake_data(handshake_data& hshd); bool merge_peerlist_with_local(const std::vector<peerlist_entry>& bs); @@ -216,7 +337,7 @@ namespace nodetool bool do_peer_timed_sync(const epee::net_utils::connection_context_base& context, peerid_type peer_id); bool make_new_connection_from_anchor_peerlist(const std::vector<anchor_peerlist_entry>& anchor_peerlist); - bool make_new_connection_from_peerlist(bool use_white_list); + bool make_new_connection_from_peerlist(network_zone& zone, bool use_white_list); bool try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist = false, uint64_t last_seen_stamp = 0, PeerType peer_type = white, uint64_t first_seen_stamp = 0); size_t get_random_index_with_fixed_probability(size_t max_index); bool is_peer_used(const peerlist_entry& peer); @@ -227,7 +348,7 @@ namespace nodetool template<class t_callback> bool try_ping(basic_node_data& node_data, p2p_connection_context& context, const t_callback &cb); bool try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f); - bool make_expected_connections_count(PeerType peer_type, size_t expected_connections); + bool make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections); void cache_connect_fail_info(const epee::net_utils::network_address& addr); bool is_addr_recently_failed(const epee::net_utils::network_address& addr); bool is_priority_node(const epee::net_utils::network_address& na); @@ -240,14 +361,8 @@ namespace nodetool template <class Container> bool parse_peers_and_add_to_container(const boost::program_options::variables_map& vm, const command_line::arg_descriptor<std::vector<std::string> > & arg, Container& container); - bool set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_out_peers() const { return m_config.m_net_config.max_out_connection_count; } - bool get_current_out_peers() const { return m_current_number_of_out_peers; } - - bool set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_in_peers() const { return m_config.m_net_config.max_in_connection_count; } - bool get_current_in_peers() const { return m_current_number_of_in_peers; } - + bool set_max_out_peers(network_zone& zone, int64_t max); + bool set_max_in_peers(network_zone& zone, int64_t max); bool set_tos_flag(const boost::program_options::variables_map& vm, int limit); bool set_rate_up_limit(const boost::program_options::variables_map& vm, int64_t limit); @@ -255,6 +370,11 @@ namespace nodetool bool set_rate_limit(const boost::program_options::variables_map& vm, int64_t limit); bool has_too_many_connections(const epee::net_utils::network_address &address); + uint64_t get_connections_count(); + size_t get_incoming_connections_count(); + size_t get_incoming_connections_count(network_zone&); + size_t get_outgoing_connections_count(); + size_t get_outgoing_connections_count(network_zone&); bool check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp); bool gray_peerlist_housekeeping(); @@ -272,25 +392,7 @@ namespace nodetool std::string print_connections_container(); - typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context> > net_server; - - struct config - { - network_config m_net_config; - uint64_t m_peer_id; - uint32_t m_support_flags; - - BEGIN_KV_SERIALIZE_MAP() - KV_SERIALIZE(m_net_config) - KV_SERIALIZE(m_peer_id) - KV_SERIALIZE(m_support_flags) - END_KV_SERIALIZE_MAP() - }; - public: - config m_config; // TODO was private, add getters? - std::atomic<unsigned int> m_current_number_of_out_peers; - std::atomic<unsigned int> m_current_number_of_in_peers; void set_save_graph(bool save_graph) { @@ -304,7 +406,6 @@ namespace nodetool bool m_first_connection_maker_call; uint32_t m_listening_port; uint32_t m_external_port; - uint32_t m_ip_address; bool m_allow_local_ip; bool m_hide_my_port; bool m_no_igd; @@ -316,7 +417,7 @@ namespace nodetool //connections_indexed_container m_connections; t_payload_net_handler& m_payload_handler; - peerlist_manager m_peerlist; + peerlist_storage m_peerlist_storage; epee::math_helper::once_a_time_seconds<P2P_DEFAULT_HANDSHAKE_INTERVAL> m_peer_handshake_idle_maker_interval; epee::math_helper::once_a_time_seconds<1> m_connections_maker_interval; @@ -324,8 +425,6 @@ namespace nodetool epee::math_helper::once_a_time_seconds<60> m_gray_peerlist_housekeeping_interval; epee::math_helper::once_a_time_seconds<3600, false> m_incoming_connections_interval; - std::string m_bind_ip; - std::string m_port; #ifdef ALLOW_DEBUG_COMMANDS uint64_t m_last_stat_request_time; #endif @@ -333,11 +432,22 @@ namespace nodetool std::vector<epee::net_utils::network_address> m_exclusive_peers; std::vector<epee::net_utils::network_address> m_seed_nodes; bool m_fallback_seed_nodes_added; - std::list<nodetool::peerlist_entry> m_command_line_peers; + std::vector<nodetool::peerlist_entry> m_command_line_peers; uint64_t m_peer_livetime; //keep connections to initiate some interactions - net_server m_net_server; - boost::uuids::uuid m_network_id; + + + static boost::optional<p2p_connection_context> public_connect(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t); + static boost::optional<p2p_connection_context> socks_connect(network_zone&, epee::net_utils::network_address const&, epee::net_utils::ssl_support_t); + + + /* A `std::map` provides constant iterators and key/value pointers even with + inserts/erases to _other_ elements. This makes the configuration step easier + since references can safely be stored on the stack. Do not insert/erase + after configuration and before destruction, lock safety would need to be + added. `std::map::operator[]` WILL insert! */ + std::map<epee::net_utils::zone, network_zone> m_network_zones; + std::map<epee::net_utils::network_address, time_t> m_conn_fails_cache; epee::critical_section m_conn_fails_cache_lock; @@ -351,7 +461,10 @@ namespace nodetool boost::mutex m_used_stripe_peers_mutex; std::array<std::list<epee::net_utils::network_address>, 1 << CRYPTONOTE_PRUNING_LOG_STRIPES> m_used_stripe_peers; + boost::uuids::uuid m_network_id; cryptonote::network_type m_nettype; + + epee::net_utils::ssl_support_t m_ssl_support; }; const int64_t default_limit_up = P2P_DEFAULT_LIMIT_RATE_UP; // kB/s @@ -364,6 +477,8 @@ namespace nodetool extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_priority_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_proxy; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound; extern const command_line::arg_descriptor<bool> arg_p2p_hide_my_port; extern const command_line::arg_descriptor<bool> arg_no_igd; @@ -380,3 +495,4 @@ namespace nodetool } POP_WARNINGS + diff --git a/src/p2p/net_node.inl b/src/p2p/net_node.inl index c594984d4..e3d804086 100644 --- a/src/p2p/net_node.inl +++ b/src/p2p/net_node.inl @@ -31,25 +31,34 @@ // IP blocking adapted from Boolberry #include <algorithm> +#include <boost/bind.hpp> #include <boost/date_time/posix_time/posix_time.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/optional/optional.hpp> #include <boost/thread/thread.hpp> #include <boost/uuid/uuid_io.hpp> -#include <boost/bind.hpp> #include <atomic> +#include <functional> +#include <limits> +#include <memory> +#include <tuple> +#include <vector> #include "version.h" #include "string_tools.h" #include "common/util.h" #include "common/dns_utils.h" #include "common/pruning.h" +#include "net/error.h" #include "net/net_helper.h" #include "math_helper.h" +#include "misc_log_ex.h" #include "p2p_protocol_defs.h" -#include "net_peerlist_boost_serialization.h" #include "net/local_ip.h" #include "crypto/crypto.h" #include "storages/levin_abstract_invoke2.h" #include "cryptonote_core/cryptonote_core.h" +#include "net/parse.h" #include <miniupnp/miniupnpc/miniupnpc.h> #include <miniupnp/miniupnpc/upnpcommands.h> @@ -64,6 +73,20 @@ namespace nodetool { + template<class t_payload_net_handler> + node_server<t_payload_net_handler>::~node_server() + { + // tcp server uses io_service in destructor, and every zone uses + // io_service from public zone. + for (auto current = m_network_zones.begin(); current != m_network_zones.end(); /* below */) + { + if (current->first != epee::net_utils::zone::public_) + current = m_network_zones.erase(current); + else + ++current; + } + } + //----------------------------------------------------------------------------------- inline bool append_net_address(std::vector<epee::net_utils::network_address> & seed_nodes, std::string const & addr, uint16_t default_port); //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -77,6 +100,8 @@ namespace nodetool command_line::add_arg(desc, arg_p2p_add_priority_node); command_line::add_arg(desc, arg_p2p_add_exclusive_node); command_line::add_arg(desc, arg_p2p_seed_node); + command_line::add_arg(desc, arg_proxy); + command_line::add_arg(desc, arg_anonymous_inbound); command_line::add_arg(desc, arg_p2p_hide_my_port); command_line::add_arg(desc, arg_no_igd); command_line::add_arg(desc, arg_out_peers); @@ -91,84 +116,14 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init_config() { - // TRY_ENTRY(); - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ifstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - // first try reading in portable mode - boost::archive::portable_binary_iarchive a(p2p_data); - a >> *this; - } - catch (...) - { - // if failed, try reading in unportable mode - boost::filesystem::copy_file(state_file_path, state_file_path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); - p2p_data.close(); - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - boost::archive::binary_iarchive a(p2p_data); - a >> *this; - } - catch (const std::exception &e) - { - MWARNING("Failed to load p2p config file, falling back to default config"); - m_peerlist = peerlist_manager(); // it was probably half clobbered by the failed load - make_default_config(); - } - } - else - { - make_default_config(); - } - } - }else - { - make_default_config(); - } - -#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED - std::list<peerlist_entry> plw; - while (m_peerlist.get_white_peers_count()) - { - plw.push_back(peerlist_entry()); - m_peerlist.get_white_peer_by_index(plw.back(), 0); - m_peerlist.remove_from_peer_white(plw.back()); - } - for (auto &e:plw) - m_peerlist.append_with_peer_white(e); - - std::list<peerlist_entry> plg; - while (m_peerlist.get_gray_peers_count()) - { - plg.push_back(peerlist_entry()); - m_peerlist.get_gray_peer_by_index(plg.back(), 0); - m_peerlist.remove_from_peer_gray(plg.back()); - } - for (auto &e:plg) - m_peerlist.append_with_peer_gray(e); -#endif - - // always recreate a new peer id - make_default_peer_id(); - - //at this moment we have hardcoded config - m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; - m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; //20 MB limit - m_config.m_net_config.config_id = 0; // initial config - m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; - m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; - m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; - m_config.m_support_flags = P2P_SUPPORT_FLAGS; + auto storage = peerlist_storage::open(m_config_folder + "/" + P2P_NET_DATA_FILENAME); + if (storage) + m_peerlist_storage = std::move(*storage); + m_network_zones[epee::net_utils::zone::public_].m_config.m_support_flags = P2P_SUPPORT_FLAGS; m_first_connection_maker_call = true; + CATCH_ENTRY_L0("node_server::init_config", false); return true; } @@ -176,17 +131,26 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::for_each_connection(std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + } } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::for_connection(const boost::uuids::uuid &connection_id, std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - return m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + const bool result = zone.second.m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + if (result) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -206,36 +170,33 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_peer_id() - { - m_config.m_peer_id = crypto::rand<uint64_t>(); - return true; - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_config() - { - return make_default_peer_id(); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::block_host(const epee::net_utils::network_address &addr, time_t seconds) { + if(!addr.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); m_blocked_hosts[addr.host_str()] = time(nullptr) + seconds; - // drop any connection to that IP - std::list<boost::uuids::uuid> conns; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + // drop any connection to that address. This should only have to look into + // the zone related to the connection, but really make sure everything is + // swept ... + std::vector<boost::uuids::uuid> conns; + for(auto& zone : m_network_zones) { - if (cntxt.m_remote_address.is_same_host(addr)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - conns.push_back(cntxt.m_connection_id); - } - return true; - }); - for (const auto &c: conns) - m_net_server.get_config_object().close(c); + if (cntxt.m_remote_address.is_same_host(addr)) + { + conns.push_back(cntxt.m_connection_id); + } + return true; + }); + for (const auto &c: conns) + zone.second.m_net_server.get_config_object().close(c); + + conns.clear(); + } MCLOG_CYAN(el::Level::Info, "global", "Host " << addr.host_str() << " blocked."); return true; @@ -256,6 +217,9 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::add_host_fail(const epee::net_utils::network_address &address) { + if(!address.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_host_fails_score_lock); uint64_t fails = ++m_host_fails_score[address.host_str()]; MDEBUG("Host " << address.host_str() << " fail score=" << fails); @@ -270,12 +234,6 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::parse_peer_from_string(epee::net_utils::network_address& pe, const std::string& node_addr, uint16_t default_port) - { - return epee::net_utils::create_network_address(pe, node_addr, default_port); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::handle_command_line( const boost::program_options::variables_map& vm ) @@ -284,8 +242,11 @@ namespace nodetool bool stagenet = command_line::get_arg(vm, cryptonote::arg_stagenet_on); m_nettype = testnet ? cryptonote::TESTNET : stagenet ? cryptonote::STAGENET : cryptonote::MAINNET; - m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); - m_port = command_line::get_arg(vm, arg_p2p_bind_port); + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + public_zone.m_connect = &public_connect; + public_zone.m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); + public_zone.m_port = command_line::get_arg(vm, arg_p2p_bind_port); + public_zone.m_can_pingback = true; m_external_port = command_line::get_arg(vm, arg_p2p_external_port); m_allow_local_ip = command_line::get_arg(vm, arg_p2p_allow_local_ip); m_no_igd = command_line::get_arg(vm, arg_no_igd); @@ -299,14 +260,20 @@ namespace nodetool nodetool::peerlist_entry pe = AUTO_VAL_INIT(pe); pe.id = crypto::rand<uint64_t>(); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(pe.adr, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - m_command_line_peers.push_back(pe); + add_zone(adr->get_zone()); + pe.adr = std::move(*adr); + m_command_line_peers.push_back(std::move(pe)); continue; } + CHECK_AND_ASSERT_MES( + adr == net::error::unsupported_address, false, "Bad address (\"" << pr_str << "\"): " << adr.error().message() + ); + std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -343,10 +310,13 @@ namespace nodetool if(command_line::has_arg(vm, arg_p2p_hide_my_port)) m_hide_my_port = true; - if ( !set_max_out_peers(vm, command_line::get_arg(vm, arg_out_peers) ) ) + if ( !set_max_out_peers(public_zone, command_line::get_arg(vm, arg_out_peers) ) ) return false; + else + m_payload_handler.set_max_out_peers(public_zone.m_config.m_net_config.max_out_connection_count); - if ( !set_max_in_peers(vm, command_line::get_arg(vm, arg_in_peers) ) ) + + if ( !set_max_in_peers(public_zone, command_line::get_arg(vm, arg_in_peers) ) ) return false; if ( !set_tos_flag(vm, command_line::get_arg(vm, arg_tos_flag) ) ) @@ -361,6 +331,58 @@ namespace nodetool if ( !set_rate_limit(vm, command_line::get_arg(vm, arg_limit_rate) ) ) return false; + + auto proxies = get_proxies(vm); + if (!proxies) + return false; + + for (auto& proxy : *proxies) + { + network_zone& zone = add_zone(proxy.zone); + if (zone.m_connect != nullptr) + { + MERROR("Listed --" << arg_proxy.name << " twice with " << epee::net_utils::zone_to_string(proxy.zone)); + return false; + } + zone.m_connect = &socks_connect; + zone.m_proxy_address = std::move(proxy.address); + + if (!set_max_out_peers(zone, proxy.max_connections)) + return false; + } + + for (const auto& zone : m_network_zones) + { + if (zone.second.m_connect == nullptr) + { + MERROR("Set outgoing peer for " << epee::net_utils::zone_to_string(zone.first) << " but did not set --" << arg_proxy.name); + return false; + } + } + + auto inbounds = get_anonymous_inbounds(vm); + if (!inbounds) + return false; + + for (auto& inbound : *inbounds) + { + network_zone& zone = add_zone(inbound.our_address.get_zone()); + + if (!zone.m_bind_ip.empty()) + { + MERROR("Listed --" << arg_anonymous_inbound.name << " twice with " << epee::net_utils::zone_to_string(inbound.our_address.get_zone()) << " network"); + return false; + } + + zone.m_bind_ip = std::move(inbound.local_ip); + zone.m_port = std::move(inbound.local_port); + zone.m_net_server.set_default_remote(std::move(inbound.default_remote)); + zone.m_our_address = std::move(inbound.our_address); + + if (!set_max_in_peers(zone, inbound.max_connections)) + return false; + } + return true; } //----------------------------------------------------------------------------------- @@ -442,7 +464,17 @@ namespace nodetool } return full_addrs; } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + typename node_server<t_payload_net_handler>::network_zone& node_server<t_payload_net_handler>::add_zone(const epee::net_utils::zone zone) + { + const auto zone_ = m_network_zones.lower_bound(zone); + if (zone_ != m_network_zones.end() && zone_->first == zone) + return zone_->second; + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + return m_network_zones.emplace_hint(zone_, std::piecewise_construct, std::make_tuple(zone), std::tie(public_zone.m_net_server.get_io_service()))->second; + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init(const boost::program_options::variables_map& vm) @@ -559,45 +591,82 @@ namespace nodetool MDEBUG("Number of seed nodes: " << m_seed_nodes.size()); m_config_folder = command_line::get_arg(vm, cryptonote::arg_data_dir); + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); - if ((m_nettype == cryptonote::MAINNET && m_port != std::to_string(::config::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::TESTNET && m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::STAGENET && m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { - m_config_folder = m_config_folder + "/" + m_port; + if ((m_nettype == cryptonote::MAINNET && public_zone.m_port != std::to_string(::config::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::TESTNET && public_zone.m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::STAGENET && public_zone.m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { + m_config_folder = m_config_folder + "/" + public_zone.m_port; } res = init_config(); CHECK_AND_ASSERT_MES(res, false, "Failed to init config."); - res = m_peerlist.init(m_allow_local_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + for (auto& zone : m_network_zones) + { + res = zone.second.m_peerlist.init(m_peerlist_storage.take_zone(zone.first), m_allow_local_ip); + CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + } + for(const auto& p: m_command_line_peers) + m_network_zones.at(p.adr.get_zone()).m_peerlist.append_with_peer_white(p); - for(auto& p: m_command_line_peers) - m_peerlist.append_with_peer_white(p); +// all peers are now setup +#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED + for (auto& zone : m_network_zones) + { + std::list<peerlist_entry> plw; + while (zone.second.m_peerlist.get_white_peers_count()) + { + plw.push_back(peerlist_entry()); + zone.second.m_peerlist.get_white_peer_by_index(plw.back(), 0); + zone.second.m_peerlist.remove_from_peer_white(plw.back()); + } + for (auto &e:plw) + zone.second.m_peerlist.append_with_peer_white(e); + + std::list<peerlist_entry> plg; + while (zone.second.m_peerlist.get_gray_peers_count()) + { + plg.push_back(peerlist_entry()); + zone.second.m_peerlist.get_gray_peer_by_index(plg.back(), 0); + zone.second.m_peerlist.remove_from_peer_gray(plg.back()); + } + for (auto &e:plg) + zone.second.m_peerlist.append_with_peer_gray(e); + } +#endif //only in case if we really sure that we have external visible ip m_have_address = true; - m_ip_address = 0; m_last_stat_request_time = 0; //configure self - m_net_server.set_threads_prefix("P2P"); - m_net_server.get_config_object().set_handler(this); - m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; - m_net_server.set_connection_filter(this); + + public_zone.m_net_server.set_threads_prefix("P2P"); // all zones use these threads/asio::io_service // from here onwards, it's online stuff if (m_offline) return res; //try to bind - MINFO("Binding on " << m_bind_ip << ":" << m_port); - res = m_net_server.init_server(m_port, m_bind_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + m_ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_disabled; + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().set_handler(this); + zone.second.m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; - m_listening_port = m_net_server.get_binded_port(); - MLOG_GREEN(el::Level::Info, "Net service bound to " << m_bind_ip << ":" << m_listening_port); + if (!zone.second.m_bind_ip.empty()) + { + zone.second.m_net_server.set_connection_filter(this); + MINFO("Binding on " << zone.second.m_bind_ip << ":" << zone.second.m_port); + res = zone.second.m_net_server.init_server(zone.second.m_port, zone.second.m_bind_ip, epee::net_utils::ssl_support_t::e_ssl_support_disabled); + CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + } + } + + m_listening_port = public_zone.m_net_server.get_binded_port(); + MLOG_GREEN(el::Level::Info, "Net service bound to " << public_zone.m_bind_ip << ":" << m_listening_port); if(m_external_port) MDEBUG("External port defined as " << m_external_port); @@ -621,44 +690,45 @@ namespace nodetool mPeersLoggerThread.reset(new boost::thread([&]() { _note("Thread monitor number of peers - start"); - while (!is_closing && !m_net_server.is_stop_signal_sent()) + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + while (!is_closing && !public_zone.m_net_server.is_stop_signal_sent()) { // main loop of thread //number_of_peers = m_net_server.get_config_object().get_connections_count(); - unsigned int number_of_in_peers = 0; - unsigned int number_of_out_peers = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - if (cntxt.m_is_income) + unsigned int number_of_in_peers = 0; + unsigned int number_of_out_peers = 0; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - ++number_of_in_peers; - } - else - { - ++number_of_out_peers; - } - return true; - }); // lambda - - m_current_number_of_in_peers = number_of_in_peers; - m_current_number_of_out_peers = number_of_out_peers; - + if (cntxt.m_is_income) + { + ++number_of_in_peers; + } + else + { + ++number_of_out_peers; + } + return true; + }); // lambda + zone.second.m_current_number_of_in_peers = number_of_in_peers; + zone.second.m_current_number_of_out_peers = number_of_out_peers; + } boost::this_thread::sleep_for(boost::chrono::seconds(1)); } // main loop of thread _note("Thread monitor number of peers - done"); })); // lambda + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + public_zone.m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); + public_zone.m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); + //here you can set worker threads count int thrds_count = 10; - - m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); - m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); - boost::thread::attributes attrs; attrs.set_stack_size(THREAD_STACK_SIZE); - //go to loop MINFO("Run net_service loop( " << thrds_count << " threads)..."); - if(!m_net_server.run_server(thrds_count, true, attrs)) + if(!public_zone.m_net_server.run_server(thrds_count, true, attrs)) { LOG_ERROR("Failed to run net tcp server!"); } @@ -666,23 +736,34 @@ namespace nodetool MINFO("net_service loop stopped."); return true; } - + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + uint64_t node_server<t_payload_net_handler>::get_public_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_net_server.get_config_object().get_connections_count(); + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> uint64_t node_server<t_payload_net_handler>::get_connections_count() { - return m_net_server.get_config_object().get_connections_count(); + std::uint64_t count = 0; + for (auto& zone : m_network_zones) + count += zone.second.m_net_server.get_config_object().get_connections_count(); + return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::deinit() { kill(); - m_peerlist.deinit(); if (!m_offline) { - m_net_server.deinit_server(); + for(auto& zone : m_network_zones) + zone.second.m_net_server.deinit_server(); // remove UPnP port mapping if(!m_no_igd) delete_upnp_port_mapping(m_listening_port); @@ -693,28 +774,25 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::store_config() { - TRY_ENTRY(); + if (!tools::create_directories_if_necessary(m_config_folder)) { - MWARNING("Failed to create data directory: " << m_config_folder); + MWARNING("Failed to create data directory \"" << m_config_folder); return false; } - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ofstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); - if(p2p_data.fail()) + peerlist_types active{}; + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(active); + + const std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; + if (!m_peerlist_storage.store(state_file_path, active)) { MWARNING("Failed to save config to file " << state_file_path); return false; - }; - - boost::archive::portable_binary_oarchive a(p2p_data); - a << *this; - return true; - CATCH_ENTRY_L0("blockchain_storage::save", false); - + } + CATCH_ENTRY_L0("node_server::store", false); return true; } //----------------------------------------------------------------------------------- @@ -722,36 +800,38 @@ namespace nodetool bool node_server<t_payload_net_handler>::send_stop_signal() { MDEBUG("[node] sending stop signal"); - m_net_server.send_stop_signal(); + for (auto& zone : m_network_zones) + zone.second.m_net_server.send_stop_signal(); MDEBUG("[node] Stop signal sent"); - std::list<boost::uuids::uuid> connection_ids; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - connection_ids.push_back(cntxt.m_connection_id); - return true; - }); - for (const auto &connection_id: connection_ids) - m_net_server.get_config_object().close(connection_id); - + for (auto& zone : m_network_zones) + { + std::list<boost::uuids::uuid> connection_ids; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { + connection_ids.push_back(cntxt.m_connection_id); + return true; + }); + for (const auto &connection_id: connection_ids) + zone.second.m_net_server.get_config_object().close(connection_id); + } m_payload_handler.stop(); - return true; } //----------------------------------------------------------------------------------- - - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::do_handshake_with_peer(peerid_type& pi, p2p_connection_context& context_, bool just_take_peerlist) { + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + typename COMMAND_HANDSHAKE::request arg; typename COMMAND_HANDSHAKE::response rsp; - get_local_node_data(arg.node_data); + get_local_node_data(arg.node_data, zone); m_payload_handler.get_payload_sync_data(arg.payload_data); epee::simple_event ev; std::atomic<bool> hsh_result(false); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, m_net_server.get_config_object(), + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, zone.m_net_server.get_config_object(), [this, &pi, &ev, &hsh_result, &just_take_peerlist, &context_](int code, const typename COMMAND_HANDSHAKE::response& rsp, p2p_connection_context& context) { epee::misc_utils::auto_scope_leave_caller scope_exit_handler = epee::misc_utils::create_scope_leave_handler([&](){ev.raise();}); @@ -785,13 +865,17 @@ namespace nodetool } pi = context.peer_id = rsp.node_data.peer_id; - m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); - if(rsp.node_data.peer_id == m_config.m_peer_id) + // move + for (auto const& zone : m_network_zones) { - LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); - hsh_result = false; - return; + if(rsp.node_data.peer_id == zone.second.m_config.m_peer_id) + { + LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); + hsh_result = false; + return; + } } LOG_INFO_CC(context, "New connection handshaked, pruning seed " << epee::string_tools::to_string_hex(context.m_pruning_seed)); LOG_DEBUG_CC(context, " COMMAND_HANDSHAKE INVOKED OK"); @@ -810,7 +894,7 @@ namespace nodetool if(!hsh_result) { LOG_WARNING_CC(context_, "COMMAND_HANDSHAKE Failed"); - m_net_server.get_config_object().close(context_.m_connection_id); + m_network_zones.at(context_.m_remote_address.get_zone()).m_net_server.get_config_object().close(context_.m_connection_id); } else { @@ -829,7 +913,8 @@ namespace nodetool typename COMMAND_TIMED_SYNC::request arg = AUTO_VAL_INIT(arg); m_payload_handler.get_payload_sync_data(arg.payload_data); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, zone.m_net_server.get_config_object(), [this](int code, const typename COMMAND_TIMED_SYNC::response& rsp, p2p_connection_context& context) { context.m_in_timedsync = false; @@ -842,11 +927,11 @@ namespace nodetool if(!handle_remote_peerlist(rsp.local_peerlist_new, rsp.local_time, context)) { LOG_WARNING_CC(context, "COMMAND_TIMED_SYNC: failed to handle_remote_peerlist(...), closing connection."); - m_net_server.get_config_object().close(context.m_connection_id ); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id ); add_host_fail(context.m_remote_address); } if(!context.m_is_income) - m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); m_payload_handler.process_payload_sync_data(rsp.payload_data, context, false); }); @@ -874,53 +959,61 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const peerlist_entry& peer) { - - if(m_config.m_peer_id == peer.id) - return true;//dont make connections to ourself + for(const auto& zone : m_network_zones) + if(zone.second.m_config.m_peer_id == peer.id) + return true;//dont make connections to ourself bool used = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - used = true; - return false;//stop enumerating - } - return true; - }); + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); - return used; + if(used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const anchor_peerlist_entry& peer) { - if(m_config.m_peer_id == peer.id) { - return true;//dont make connections to ourself - } - - bool used = false; - - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) - { - used = true; - - return false;//stop enumerating + for(auto& zone : m_network_zones) { + if(zone.second.m_config.m_peer_id == peer.id) { + return true;//dont make connections to ourself } - - return true; - }); - - return used; + bool used = false; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); + if (used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_addr_connected(const epee::net_utils::network_address& peer) { + const auto zone = m_network_zones.find(peer.get_zone()); + if (zone == m_network_zones.end()) + return false; + bool connected = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone->second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if(!cntxt.m_is_income && peer == cntxt.m_remote_address) { @@ -945,47 +1038,44 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist, uint64_t last_seen_stamp, PeerType peer_type, uint64_t first_seen_stamp) { - if (m_current_number_of_out_peers == m_config.m_net_config.max_out_connection_count) // out peers limit + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) // outgoing connections in zone not possible + return false; + + if (zone.m_current_number_of_out_peers == zone.m_config.m_net_config.max_out_connection_count) // out peers limit { return false; } - else if (m_current_number_of_out_peers > m_config.m_net_config.max_out_connection_count) + else if (zone.m_current_number_of_out_peers > zone.m_config.m_net_config.max_out_connection_count) { - m_net_server.get_config_object().del_out_connections(1); - m_current_number_of_out_peers --; // atomic variable, update time = 1s + zone.m_net_server.get_config_object().del_out_connections(1); + --(zone.m_current_number_of_out_peers); // atomic variable, update time = 1s return false; } + + MDEBUG("Connecting to " << na.str() << "(peer_type=" << peer_type << ", last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = peer_type == anchor; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if(!res) + auto con = zone.m_connect(zone, na, m_ssl_support); + if(!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str() + LOG_PRINT_CC_PRIORITY_NODE(is_priority, bool(con), "Connect failed to " << na.str() /*<< ", try " << try_count*/); //m_peerlist.set_peer_unreachable(pe); return false; } + con->m_anchor = peer_type == anchor; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, just_take_peerlist); + bool res = do_handshake_with_peer(pi, *con, just_take_peerlist); if(!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str() /*<< ", try " << try_count*/); return false; @@ -993,8 +1083,8 @@ namespace nodetool if(just_take_peerlist) { - m_net_server.get_config_object().close(con.m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + zone.m_net_server.get_config_object().close(con->m_connection_id); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1004,8 +1094,8 @@ namespace nodetool time_t last_seen; time(&last_seen); pe_local.last_seen = static_cast<int64_t>(last_seen); - pe_local.pruning_seed = con.m_pruning_seed; - m_peerlist.append_with_peer_white(pe_local); + pe_local.pruning_seed = con->m_pruning_seed; + zone.m_peerlist.append_with_peer_white(pe_local); //update last seen and push it to peerlist manager anchor_peerlist_entry ape = AUTO_VAL_INIT(ape); @@ -1013,52 +1103,46 @@ namespace nodetool ape.id = pi; ape.first_seen = first_seen_stamp ? first_seen_stamp : time(nullptr); - m_peerlist.append_with_peer_anchor(ape); + zone.m_peerlist.append_with_peer_anchor(ape); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK."); return true; } template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp) { + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) + return false; + LOG_PRINT_L1("Connecting to " << na.str() << "(last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = false; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if (!res) { + auto con = zone.m_connect(zone, na, m_ssl_support); + if (!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, p2p_connection_context{}, "Connect failed to " << na.str()); return false; } + con->m_anchor = false; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, true); - + const bool res = do_handshake_with_peer(pi, *con, true); if (!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str()); return false; } - m_net_server.get_config_object().close(con.m_connection_id); + zone.m_net_server.get_config_object().close(con->m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1115,7 +1199,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(bool use_white_list) + bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(network_zone& zone, bool use_white_list) { size_t max_random_index = 0; @@ -1123,7 +1207,7 @@ namespace nodetool size_t try_count = 0; size_t rand_count = 0; - while(rand_count < (max_random_index+1)*3 && try_count < 10 && !m_net_server.is_stop_signal_sent()) + while(rand_count < (max_random_index+1)*3 && try_count < 10 && !zone.m_net_server.is_stop_signal_sent()) { ++rand_count; size_t random_index; @@ -1132,7 +1216,7 @@ namespace nodetool std::deque<size_t> filtered; const size_t limit = use_white_list ? 20 : std::numeric_limits<size_t>::max(); size_t idx = 0; - m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ + zone.m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ if (filtered.size() >= limit) return false; if (next_needed_pruning_stripe == 0 || pe.pruning_seed == 0) @@ -1159,7 +1243,7 @@ namespace nodetool for (size_t i = 0; i < filtered.size(); ++i) { peerlist_entry pe; - if (m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) + if (zone.m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) { MDEBUG("Reusing stripe " << next_needed_pruning_stripe << " peer " << pe.adr.str()); random_index = i; @@ -1173,7 +1257,7 @@ namespace nodetool CHECK_AND_ASSERT_MES(random_index < filtered.size(), false, "random_index < filtered.size() failed!!"); random_index = filtered[random_index]; - CHECK_AND_ASSERT_MES(random_index < (use_white_list ? m_peerlist.get_white_peers_count() : m_peerlist.get_gray_peers_count()), + CHECK_AND_ASSERT_MES(random_index < (use_white_list ? zone.m_peerlist.get_white_peers_count() : zone.m_peerlist.get_gray_peers_count()), false, "random_index < peers size failed!!"); if(tried_peers.count(random_index)) @@ -1181,7 +1265,7 @@ namespace nodetool tried_peers.insert(random_index); peerlist_entry pe = AUTO_VAL_INIT(pe); - bool r = use_white_list ? m_peerlist.get_white_peer_by_index(pe, random_index):m_peerlist.get_gray_peer_by_index(pe, random_index); + bool r = use_white_list ? zone.m_peerlist.get_white_peer_by_index(pe, random_index):zone.m_peerlist.get_gray_peer_by_index(pe, random_index); CHECK_AND_ASSERT_MES(r, false, "Failed to get random peer from peerlist(white:" << use_white_list << ")"); ++try_count; @@ -1224,9 +1308,10 @@ namespace nodetool size_t try_count = 0; size_t current_index = crypto::rand<size_t>()%m_seed_nodes.size(); + const net_server& server = m_network_zones.at(epee::net_utils::zone::public_).m_net_server; while(true) { - if(m_net_server.is_stop_signal_sent()) + if(server.is_stop_signal_sent()) return false; if(try_to_connect_and_handshake_with_new_peer(m_seed_nodes[current_index], true)) @@ -1265,13 +1350,17 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::connections_maker() { + using zone_type = epee::net_utils::zone; + if (m_offline) return true; if (!connect_to_peerlist(m_exclusive_peers)) return false; if (!m_exclusive_peers.empty()) return true; - size_t start_conn_count = get_outgoing_connections_count(); - if(!m_peerlist.get_white_peers_count() && m_seed_nodes.size()) + // Only have seeds in the public zone right now. + + size_t start_conn_count = get_public_outgoing_connections_count(); + if(!get_public_white_peers_count() && m_seed_nodes.size()) { if (!connect_to_seed()) return false; @@ -1279,38 +1368,41 @@ namespace nodetool if (!connect_to_peerlist(m_priority_peers)) return false; - size_t base_expected_white_connections = (m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; - - size_t conn_count = get_outgoing_connections_count(); - while(conn_count < m_config.m_net_config.max_out_connection_count) + for(auto& zone : m_network_zones) { - const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? m_config.m_net_config.max_out_connection_count : base_expected_white_connections; - if(conn_count < expected_white_connections) - { - //start from anchor list - while (get_outgoing_connections_count() < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT - && make_expected_connections_count(anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); - //then do white list - while (get_outgoing_connections_count() < expected_white_connections - && make_expected_connections_count(white, expected_white_connections)); - //then do grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - }else + size_t base_expected_white_connections = (zone.second.m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; + + size_t conn_count = get_outgoing_connections_count(zone.second); + while(conn_count < zone.second.m_config.m_net_config.max_out_connection_count) { - //start from grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - //and then do white list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(white, m_config.m_net_config.max_out_connection_count)); + const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? zone.second.m_config.m_net_config.max_out_connection_count : base_expected_white_connections; + if(conn_count < expected_white_connections) + { + //start from anchor list + while (get_outgoing_connections_count(zone.second) < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT + && make_expected_connections_count(zone.second, anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); + //then do white list + while (get_outgoing_connections_count(zone.second) < expected_white_connections + && make_expected_connections_count(zone.second, white, expected_white_connections)); + //then do grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + }else + { + //start from grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + //and then do white list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, white, zone.second.m_config.m_net_config.max_out_connection_count)); + } + if(zone.second.m_net_server.is_stop_signal_sent()) + return false; + conn_count = get_outgoing_connections_count(zone.second); } - if(m_net_server.is_stop_signal_sent()) - return false; - conn_count = get_outgoing_connections_count(); } - if (start_conn_count == get_outgoing_connections_count() && start_conn_count < m_config.m_net_config.max_out_connection_count) + if (start_conn_count == get_public_outgoing_connections_count() && start_conn_count < m_network_zones.at(zone_type::public_).m_config.m_net_config.max_out_connection_count) { MINFO("Failed to connect to any, trying seeds"); if (!connect_to_seed()) @@ -1321,7 +1413,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_expected_connections_count(PeerType peer_type, size_t expected_connections) + bool node_server<t_payload_net_handler>::make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections) { if (m_offline) return false; @@ -1329,14 +1421,14 @@ namespace nodetool std::vector<anchor_peerlist_entry> apl; if (peer_type == anchor) { - m_peerlist.get_and_empty_anchor_peerlist(apl); + zone.m_peerlist.get_and_empty_anchor_peerlist(apl); } - size_t conn_count = get_outgoing_connections_count(); + size_t conn_count = get_outgoing_connections_count(zone); //add new connections from white peers if(conn_count < expected_connections) { - if(m_net_server.is_stop_signal_sent()) + if(zone.m_net_server.is_stop_signal_sent()) return false; MDEBUG("Making expected connection, type " << peer_type << ", " << conn_count << "/" << expected_connections << " connections"); @@ -1345,47 +1437,104 @@ namespace nodetool return false; } - if (peer_type == white && !make_new_connection_from_peerlist(true)) { + if (peer_type == white && !make_new_connection_from_peerlist(zone, true)) { return false; } - if (peer_type == gray && !make_new_connection_from_peerlist(false)) { + if (peer_type == gray && !make_new_connection_from_peerlist(zone, false)) { return false; } } return true; } - //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + size_t node_server<t_payload_net_handler>::get_public_outgoing_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return get_outgoing_connections_count(public_zone->second); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(!cntxt.m_is_income) + if(cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(cntxt.m_is_income) + if(!cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + { + size_t count = 0; + for(auto& zone : m_network_zones) + count += get_outgoing_connections_count(zone.second); + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + { + size_t count = 0; + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.m_is_income) + ++count; + return true; + }); + } + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_white_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_white_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_gray_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_gray_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + void node_server<t_payload_net_handler>::get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white) + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + public_zone->second.m_peerlist.get_peerlist(gray, white); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::idle_worker() { m_peer_handshake_idle_maker_interval.do_call(boost::bind(&node_server<t_payload_net_handler>::peer_sync_idle_maker, this)); @@ -1401,9 +1550,11 @@ namespace nodetool { if (m_offline) return true; - if (get_incoming_connections_count() == 0) + + const auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end() && get_incoming_connections_count(public_zone->second) == 0) { - if (m_hide_my_port || m_config.m_net_config.max_in_connection_count == 0) + if (m_hide_my_port || public_zone->second.m_config.m_net_config.max_in_connection_count == 0) { MGINFO("Incoming connections disabled, enable them for full connectivity"); } @@ -1422,15 +1573,18 @@ namespace nodetool MDEBUG("STARTED PEERLIST IDLE HANDSHAKE"); typedef std::list<std::pair<epee::net_utils::connection_context_base, peerid_type> > local_connects_type; local_connects_type cncts; - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id && !cntxt.m_in_timedsync) + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) { - cntxt.m_in_timedsync = true; - cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections - } - return true; - }); + if(cntxt.peer_id && !cntxt.m_in_timedsync) + { + cntxt.m_in_timedsync = true; + cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections + } + return true; + }); + } std::for_each(cncts.begin(), cncts.end(), [&](const typename local_connects_type::value_type& vl){do_peer_timed_sync(vl.first, vl.second);}); @@ -1468,19 +1622,30 @@ namespace nodetool std::vector<peerlist_entry> peerlist_ = peerlist; if(!fix_time_delta(peerlist_, local_time, delta)) return false; + + const epee::net_utils::zone zone = context.m_remote_address.get_zone(); + for(const auto& peer : peerlist_) + { + if(peer.adr.get_zone() != zone) + { + MWARNING(context << " sent peerlist from another zone, dropping"); + return false; + } + } + LOG_DEBUG_CC(context, "REMOTE PEERLIST: TIME_DELTA: " << delta << ", remote peerlist size=" << peerlist_.size()); LOG_DEBUG_CC(context, "REMOTE PEERLIST: " << print_peerlist_to_string(peerlist_)); - return m_peerlist.merge_peerlist(peerlist_); + return m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.merge_peerlist(peerlist_); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data) + bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data, const network_zone& zone) { time_t local_time; time(&local_time); - node_data.local_time = local_time; - node_data.peer_id = m_config.m_peer_id; - if(!m_hide_my_port) + node_data.local_time = local_time; // \TODO This can be an identifying value across zones (public internet to tor/i2p) ... + node_data.peer_id = zone.m_config.m_peer_id; + if(!m_hide_my_port && zone.m_can_pingback) node_data.my_port = m_external_port ? m_external_port : m_listening_port; else node_data.my_port = 0; @@ -1490,7 +1655,7 @@ namespace nodetool //----------------------------------------------------------------------------------- #ifdef ALLOW_DEBUG_COMMANDS template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr) + bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr, const epee::net_utils::zone zone_type) { uint64_t local_time = time(NULL); uint64_t time_delata = local_time > tr.time ? local_time - tr.time: tr.time - local_time; @@ -1504,9 +1669,11 @@ namespace nodetool MWARNING("check_trust failed to check time conditions, last_stat_request_time=" << m_last_stat_request_time << ", proof_time=" << tr.time); return false; } - if(m_config.m_peer_id != tr.peer_id) + + const network_zone& zone = m_network_zones.at(zone_type); + if(zone.m_config.m_peer_id != tr.peer_id) { - MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << m_config.m_peer_id<< ")"); + MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << zone.m_config.m_peer_id<< ")"); return false; } crypto::public_key pk = AUTO_VAL_INIT(pk); @@ -1525,12 +1692,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_stat_info(int command, typename COMMAND_REQUEST_STAT_INFO::request& arg, typename COMMAND_REQUEST_STAT_INFO::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - rsp.connections_count = m_net_server.get_config_object().get_connections_count(); + rsp.connections_count = get_connections_count(); rsp.incoming_connections_count = rsp.connections_count - get_outgoing_connections_count(); rsp.version = MONERO_VERSION_FULL; rsp.os_version = tools::get_os_version_string(); @@ -1541,12 +1708,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_network_state(int command, COMMAND_REQUEST_NETWORK_STATE::request& arg, COMMAND_REQUEST_NETWORK_STATE::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { connection_entry ce; ce.adr = cntxt.m_remote_address; @@ -1556,8 +1723,9 @@ namespace nodetool return true; }); - m_peerlist.get_peerlist_full(rsp.local_peerlist_gray, rsp.local_peerlist_white); - rsp.my_id = m_config.m_peer_id; + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + zone.m_peerlist.get_peerlist(rsp.local_peerlist_gray, rsp.local_peerlist_white); + rsp.my_id = zone.m_config.m_peer_id; rsp.local_time = time(NULL); return 1; } @@ -1565,7 +1733,7 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_peer_id(int command, COMMAND_REQUEST_PEER_ID::request& arg, COMMAND_REQUEST_PEER_ID::response& rsp, p2p_connection_context& context) { - rsp.my_id = m_config.m_peer_id; + rsp.my_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } #endif @@ -1573,40 +1741,42 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_support_flags(int command, COMMAND_REQUEST_SUPPORT_FLAGS::request& arg, COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context) { - rsp.support_flags = m_config.m_support_flags; + rsp.support_flags = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_support_flags; return 1; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> void node_server<t_payload_net_handler>::request_callback(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().request_callback(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().request_callback(context.m_connection_id); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections) + bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { + std::sort(connections.begin(), connections.end()); + auto zone = m_network_zones.begin(); for(const auto& c_id: connections) { - m_net_server.get_config_object().notify(command, data_buff, c_id); + for (;;) + { + if (zone == m_network_zones.end()) + { + MWARNING("Unable to relay all messages, " << epee::net_utils::zone_to_string(c_id.first) << " not available"); + return false; + } + if (c_id.first <= zone->first) + break; + + ++zone; + } + if (zone->first == c_id.first) + zone->second.m_net_server.get_config_object().notify(command, data_buff, c_id.second); } return true; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) - { - std::list<boost::uuids::uuid> connections; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id && context.m_connection_id != cntxt.m_connection_id) - connections.push_back(cntxt.m_connection_id); - return true; - }); - return relay_notify_to_list(command, data_buff, connections); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> void node_server<t_payload_net_handler>::callback(p2p_connection_context& context) { m_payload_handler.on_callback(context); @@ -1615,21 +1785,29 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::drop_connection(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().close(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id); return true; } //----------------------------------------------------------------------------------- @@ -1639,18 +1817,21 @@ namespace nodetool if(!node_data.my_port) return false; - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), false, "Only IPv4 addresses are supported here"); const epee::net_utils::network_address na = context.m_remote_address; uint32_t actual_ip = na.as<const epee::net_utils::ipv4_network_address>().ip(); - if(!m_peerlist.is_host_allowed(context.m_remote_address)) + network_zone& zone = m_network_zones.at(na.get_zone()); + + if(!zone.m_peerlist.is_host_allowed(context.m_remote_address)) return false; + std::string ip = epee::string_tools::get_ip_string_from_int32(actual_ip); std::string port = epee::string_tools::num_to_string_fast(node_data.my_port); epee::net_utils::network_address address{epee::net_utils::ipv4_network_address(actual_ip, node_data.my_port)}; peerid_type pr = node_data.peer_id; - bool r = m_net_server.connect_async(ip, port, m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( + bool r = zone.m_net_server.connect_async(ip, port, zone.m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( const typename net_server::t_connection_context& ping_context, const boost::system::error_code& ec)->bool { @@ -1670,7 +1851,9 @@ namespace nodetool // GCC 5.1.0 gives error with second use of uint64_t (peerid_type) variable. peerid_type pr_ = pr; - bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(address.get_zone()); + + bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, zone.m_net_server.get_config_object(), [=](int code, const COMMAND_PING::response& rsp, p2p_connection_context& context) { if(code <= 0) @@ -1679,20 +1862,21 @@ namespace nodetool return; } + network_zone& zone = m_network_zones.at(address.get_zone()); if(rsp.status != PING_OK_RESPONSE_STATUS_TEXT || pr != rsp.peer_id) { LOG_WARNING_CC(ping_context, "back ping invoke wrong response \"" << rsp.status << "\" from" << address.str() << ", hsh_peer_id=" << pr_ << ", rsp.peer_id=" << rsp.peer_id); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return; } - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); cb(); }); if(!inv_call_res) { LOG_WARNING_CC(ping_context, "back ping invoke failed to " << address.str()); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return false; } return true; @@ -1707,13 +1891,16 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f) { + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + return false; + COMMAND_REQUEST_SUPPORT_FLAGS::request support_flags_request; bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_REQUEST_SUPPORT_FLAGS::response> ( context.m_connection_id, COMMAND_REQUEST_SUPPORT_FLAGS::ID, support_flags_request, - m_net_server.get_config_object(), + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object(), [=](int code, const typename COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context_) { if(code < 0) @@ -1742,8 +1929,24 @@ namespace nodetool //fill response rsp.local_time = time(NULL); - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + + const epee::net_utils::zone zone_type = context.m_remote_address.get_zone(); + network_zone& zone = m_network_zones.at(zone_type); + + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); m_payload_handler.get_payload_sync_data(rsp.payload_data); + + /* Tor/I2P nodes receiving connections via forwarding (from tor/i2p daemon) + do not know the address of the connecting peer. This is relayed to them, + iff the node has setup an inbound hidden service. The other peer will have + to use the random peer_id value to link the two. My initial thought is that + the inbound peer should leave the other side marked as `<unknown tor host>`, + etc., because someone could give faulty addresses over Tor/I2P to get the + real peer with that identity banned/blacklisted. */ + + if(!context.m_is_income && zone.m_our_address.get_zone() == zone_type) + rsp.local_peerlist_new.push_back(peerlist_entry{zone.m_our_address, zone.m_config.m_peer_id, std::time(nullptr)}); + LOG_DEBUG_CC(context, "COMMAND_TIMED_SYNC"); return 1; } @@ -1775,7 +1978,9 @@ namespace nodetool return 1; } - if (m_current_number_of_in_peers >= m_config.m_net_config.max_in_connection_count) // in peers limit + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + + if (zone.m_current_number_of_in_peers >= zone.m_config.m_net_config.max_in_connection_count) // in peers limit { LOG_WARNING_CC(context, "COMMAND_HANDSHAKE came, but already have max incoming connections, so dropping this one."); drop_connection(context); @@ -1800,14 +2005,14 @@ namespace nodetool context.peer_id = arg.node_data.peer_id; context.m_in_timedsync = false; - if(arg.node_data.peer_id != m_config.m_peer_id && arg.node_data.my_port) + if(arg.node_data.peer_id != zone.m_config.m_peer_id && arg.node_data.my_port && zone.m_can_pingback) { peerid_type peer_id_l = arg.node_data.peer_id; uint32_t port_l = arg.node_data.my_port; //try ping to be sure that we can add this peer to peer_list try_ping(arg.node_data, context, [peer_id_l, port_l, context, this]() { - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, void(), + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), void(), "Only IPv4 addresses are supported here"); //called only(!) if success pinged, update local peerlist peerlist_entry pe; @@ -1818,7 +2023,7 @@ namespace nodetool pe.last_seen = static_cast<int64_t>(last_seen); pe.id = peer_id_l; pe.pruning_seed = context.m_pruning_seed; - this->m_peerlist.append_with_peer_white(pe); + this->m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.append_with_peer_white(pe); LOG_DEBUG_CC(context, "PING SUCCESS " << context.m_remote_address.host_str() << ":" << port_l); }); } @@ -1829,8 +2034,8 @@ namespace nodetool }); //fill response - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); - get_local_node_data(rsp.node_data); + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + get_local_node_data(rsp.node_data, zone); m_payload_handler.get_payload_sync_data(rsp.payload_data); LOG_DEBUG_CC(context, "COMMAND_HANDSHAKE"); return 1; @@ -1841,7 +2046,7 @@ namespace nodetool { LOG_DEBUG_CC(context, "COMMAND_PING"); rsp.status = PING_OK_RESPONSE_STATUS_TEXT; - rsp.peer_id = m_config.m_peer_id; + rsp.peer_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } //----------------------------------------------------------------------------------- @@ -1850,7 +2055,8 @@ namespace nodetool { std::vector<peerlist_entry> pl_white; std::vector<peerlist_entry> pl_gray; - m_peerlist.get_peerlist_full(pl_gray, pl_white); + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(pl_gray, pl_white); MINFO(ENDL << "Peerlist white:" << ENDL << print_peerlist_to_string(pl_white) << ENDL << "Peerlist gray:" << ENDL << print_peerlist_to_string(pl_gray) ); return true; } @@ -1867,14 +2073,17 @@ namespace nodetool { std::stringstream ss; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - ss << cntxt.m_remote_address.str() - << " \t\tpeer_id " << cntxt.peer_id - << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") - << std::endl; - return true; - }); + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + ss << cntxt.m_remote_address.str() + << " \t\tpeer_id " << cntxt.peer_id + << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") + << std::endl; + return true; + }); + } std::string s = ss.str(); return s; } @@ -1888,11 +2097,12 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::on_connection_close(p2p_connection_context& context) { - if (!m_net_server.is_stop_signal_sent() && !context.m_is_income) { + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + if (!zone.m_net_server.is_stop_signal_sent() && !context.m_is_income) { epee::net_utils::network_address na = AUTO_VAL_INIT(na); na = context.m_remote_address; - m_peerlist.remove_from_peer_anchor(na); + zone.m_peerlist.remove_from_peer_anchor(na); } m_payload_handler.on_connection_close(context); @@ -1909,9 +2119,10 @@ namespace nodetool template<class t_payload_net_handler> template <class Container> bool node_server<t_payload_net_handler>::connect_to_peerlist(const Container& peers) { + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); for(const epee::net_utils::network_address& na: peers) { - if(m_net_server.is_stop_signal_sent()) + if(public_zone.m_net_server.is_stop_signal_sent()) return false; if(is_addr_connected(na)) @@ -1930,16 +2141,16 @@ namespace nodetool for(const std::string& pr_str: perrs) { - epee::net_utils::network_address na = AUTO_VAL_INIT(na); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(na, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - container.push_back(na); + add_zone(adr->get_zone()); + container.push_back(std::move(*adr)); continue; } std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -1951,32 +2162,47 @@ namespace nodetool } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_out_peers(network_zone& zone, int64_t max) { - if(max == -1) - max = P2P_DEFAULT_CONNECTIONS_COUNT; - m_config.m_net_config.max_out_connection_count = max; - m_payload_handler.set_max_out_peers(max); + if(max == -1) { + zone.m_config.m_net_config.max_out_connection_count = P2P_DEFAULT_CONNECTIONS_COUNT; + return true; + } + zone.m_config.m_net_config.max_out_connection_count = max; return true; } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_in_peers(network_zone& zone, int64_t max) { - m_config.m_net_config.max_in_connection_count = max; + zone.m_config.m_net_config.max_in_connection_count = max; return true; } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_out_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_out_public_peers(size_t count) { - m_net_server.get_config_object().del_out_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_out_connection_count; + public_zone->second.m_config.m_net_config.max_out_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_out_connections(current - count); + } } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_in_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_in_public_peers(size_t count) { - m_net_server.get_config_object().del_in_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_in_connection_count; + public_zone->second.m_config.m_net_config.max_in_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_in_connections(current - count); + } } template<class t_payload_net_handler> @@ -2049,10 +2275,13 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::has_too_many_connections(const epee::net_utils::network_address &address) { + if (address.get_zone() != epee::net_utils::zone::public_) + return false; // Unable to determine how many connections from host + const size_t max_connections = 1; size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if (cntxt.m_is_income && cntxt.m_remote_address.is_same_host(address)) { count++; @@ -2075,29 +2304,29 @@ namespace nodetool if (!m_exclusive_peers.empty()) return true; if (m_payload_handler.needs_new_sync_connections()) return true; - peerlist_entry pe = AUTO_VAL_INIT(pe); - - if (m_net_server.is_stop_signal_sent()) - return false; - - if (!m_peerlist.get_random_gray_peer(pe)) { - return true; - } - - bool success = check_connection_and_handshake_with_peer(pe.adr, pe.last_seen); + for (auto& zone : m_network_zones) + { + if (zone.second.m_net_server.is_stop_signal_sent()) + return false; - if (!success) { - m_peerlist.remove_from_peer_gray(pe); + if (zone.second.m_connect == nullptr) + continue; - LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + peerlist_entry pe{}; + if (!zone.second.m_peerlist.get_random_gray_peer(pe)) + continue; - return true; + if (!check_connection_and_handshake_with_peer(pe.adr, pe.last_seen)) + { + zone.second.m_peerlist.remove_from_peer_gray(pe); + LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } + else + { + zone.second.m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); + LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } } - - m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); - - LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); - return true; } @@ -2225,4 +2454,37 @@ namespace nodetool MINFO("No IGD was found."); } } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::socks_connect(network_zone& zone, const epee::net_utils::network_address& remote, epee::net_utils::ssl_support_t ssl_support) + { + auto result = socks_connect_internal(zone.m_net_server.get_stop_signal(), zone.m_net_server.get_io_service(), zone.m_proxy_address, remote); + if (result) // if no error + { + p2p_connection_context context{}; + if (zone.m_net_server.add_connection(context, std::move(*result), remote, ssl_support)) + return {std::move(context)}; + } + return boost::none; + } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::public_connect(network_zone& zone, epee::net_utils::network_address const& na, epee::net_utils::ssl_support_t ssl_support) + { + CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), boost::none, + "Only IPv4 addresses are supported here"); + const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); + + typename net_server::t_connection_context con{}; + const bool res = zone.m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), + epee::string_tools::num_to_string_fast(ipv4.port()), + zone.m_config.m_net_config.connection_timeout, + con, "0.0.0.0", ssl_support); + + if (res) + return {std::move(con)}; + return boost::none; + } } diff --git a/src/p2p/net_node_common.h b/src/p2p/net_node_common.h index 075c18ef9..944bf48e4 100644 --- a/src/p2p/net_node_common.h +++ b/src/p2p/net_node_common.h @@ -31,6 +31,8 @@ #pragma once #include <boost/uuid/uuid.hpp> +#include <utility> +#include <vector> #include "net/net_utils_base.h" #include "p2p_protocol_defs.h" @@ -43,13 +45,13 @@ namespace nodetool template<class t_connection_context> struct i_p2p_endpoint { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections)=0; - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context)=0; + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections)=0; virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool drop_connection(const epee::net_utils::connection_context_base& context)=0; virtual void request_callback(const epee::net_utils::connection_context_base& context)=0; - virtual uint64_t get_connections_count()=0; + virtual uint64_t get_public_connections_count()=0; + virtual size_t get_zone_count() const=0; virtual void for_each_connection(std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool for_connection(const boost::uuids::uuid&, std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool block_host(const epee::net_utils::network_address &address, time_t seconds = 0)=0; @@ -64,11 +66,7 @@ namespace nodetool template<class t_connection_context> struct p2p_endpoint_stub: public i_p2p_endpoint<t_connection_context> { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections) - { - return false; - } - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { return false; } @@ -97,7 +95,12 @@ namespace nodetool return false; } - virtual uint64_t get_connections_count() + virtual size_t get_zone_count() const + { + return 0; + } + + virtual uint64_t get_public_connections_count() { return false; } diff --git a/src/p2p/net_peerlist.cpp b/src/p2p/net_peerlist.cpp new file mode 100644 index 000000000..ce5c67fe5 --- /dev/null +++ b/src/p2p/net_peerlist.cpp @@ -0,0 +1,295 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "net_peerlist.h" + +#include <algorithm> +#include <functional> +#include <fstream> +#include <iterator> + +#include <boost/archive/binary_iarchive.hpp> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/range/join.hpp> +#include <boost/serialization/version.hpp> + +#include "net_peerlist_boost_serialization.h" + + +namespace nodetool +{ + namespace + { + constexpr unsigned CURRENT_PEERLIST_STORAGE_ARCHIVE_VER = 6; + + struct by_zone + { + using zone = epee::net_utils::zone; + + template<typename T> + bool operator()(const T& left, const zone right) const + { + return left.adr.get_zone() < right; + } + + template<typename T> + bool operator()(const zone left, const T& right) const + { + return left < right.adr.get_zone(); + } + + template<typename T, typename U> + bool operator()(const T& left, const U& right) const + { + return left.adr.get_zone() < right.adr.get_zone(); + } + }; + + template<typename Elem, typename Archive> + std::vector<Elem> load_peers(Archive& a, unsigned ver) + { + // at v6, we drop existing peerlists, because annoying change + if (ver < 6) + return {}; + + uint64_t size = 0; + a & size; + + Elem ple{}; + + std::vector<Elem> elems{}; + elems.reserve(size); + while (size--) + { + a & ple; + elems.push_back(std::move(ple)); + } + + return elems; + } + + template<typename Archive, typename Range> + void save_peers(Archive& a, const Range& elems) + { + const uint64_t size = elems.size(); + a & size; + for (const auto& elem : elems) + a & elem; + } + + template<typename T> + std::vector<T> do_take_zone(std::vector<T>& src, epee::net_utils::zone zone) + { + const auto start = std::lower_bound(src.begin(), src.end(), zone, by_zone{}); + const auto end = std::upper_bound(start, src.end(), zone, by_zone{}); + + std::vector<T> out{}; + out.assign(std::make_move_iterator(start), std::make_move_iterator(end)); + src.erase(start, end); + return out; + } + + template<typename Container, typename T> + void add_peers(Container& dest, std::vector<T>&& src) + { + dest.insert(std::make_move_iterator(src.begin()), std::make_move_iterator(src.end())); + } + + template<typename Container, typename Range> + void copy_peers(Container& dest, const Range& src) + { + std::copy(src.begin(), src.end(), std::back_inserter(dest)); + } + } // anonymous + + struct peerlist_join + { + const peerlist_types& ours; + const peerlist_types& other; + }; + + template<typename Archive> + void serialize(Archive& a, peerlist_types& elem, unsigned ver) + { + elem.white = load_peers<peerlist_entry>(a, ver); + elem.gray = load_peers<peerlist_entry>(a, ver); + elem.anchor = load_peers<anchor_peerlist_entry>(a, ver); + + if (ver == 0) + { + // from v1, we do not store the peer id anymore + peerid_type peer_id{}; + a & peer_id; + } + } + + template<typename Archive> + void serialize(Archive& a, peerlist_join elem, unsigned ver) + { + save_peers(a, boost::range::join(elem.ours.white, elem.other.white)); + save_peers(a, boost::range::join(elem.ours.gray, elem.other.gray)); + save_peers(a, boost::range::join(elem.ours.anchor, elem.other.anchor)); + } + + boost::optional<peerlist_storage> peerlist_storage::open(std::istream& src, const bool new_format) + { + try + { + peerlist_storage out{}; + if (new_format) + { + boost::archive::portable_binary_iarchive a{src}; + a >> out.m_types; + } + else + { + boost::archive::binary_iarchive a{src}; + a >> out.m_types; + } + + if (src.good()) + { + std::sort(out.m_types.white.begin(), out.m_types.white.end(), by_zone{}); + std::sort(out.m_types.gray.begin(), out.m_types.gray.end(), by_zone{}); + std::sort(out.m_types.anchor.begin(), out.m_types.anchor.end(), by_zone{}); + return {std::move(out)}; + } + } + catch (const std::exception& e) + {} + + return boost::none; + } + + boost::optional<peerlist_storage> peerlist_storage::open(const std::string& path) + { + std::ifstream src_file{}; + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + boost::optional<peerlist_storage> out = open(src_file, true); + if (!out) + { + // if failed, try reading in unportable mode + boost::filesystem::copy_file(path, path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); + src_file.close(); + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + out = open(src_file, false); + if (!out) + { + // This is different from the `return boost::none` cases above. Those + // cases could fail due to bad file permissions, so a shutdown is + // likely more appropriate. + MWARNING("Failed to load p2p config file, falling back to default config"); + out.emplace(); + } + } + + return out; + } + + peerlist_storage::~peerlist_storage() noexcept + {} + + bool peerlist_storage::store(std::ostream& dest, const peerlist_types& other) const + { + try + { + boost::archive::portable_binary_oarchive a{dest}; + const peerlist_join pj{std::cref(m_types), std::cref(other)}; + a << pj; + return dest.good(); + } + catch (const boost::archive::archive_exception& e) + {} + + return false; + } + + bool peerlist_storage::store(const std::string& path, const peerlist_types& other) const + { + std::ofstream dest_file{}; + dest_file.open( path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); + if(dest_file.fail()) + return false; + + return store(dest_file, other); + } + + peerlist_types peerlist_storage::take_zone(epee::net_utils::zone zone) + { + peerlist_types out{}; + out.white = do_take_zone(m_types.white, zone); + out.gray = do_take_zone(m_types.gray, zone); + out.anchor = do_take_zone(m_types.anchor, zone); + return out; + } + + bool peerlist_manager::init(peerlist_types&& peers, bool allow_local_ip) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + + if (!m_peers_white.empty() || !m_peers_gray.empty() || !m_peers_anchor.empty()) + return false; + + add_peers(m_peers_white.get<by_addr>(), std::move(peers.white)); + add_peers(m_peers_gray.get<by_addr>(), std::move(peers.gray)); + add_peers(m_peers_anchor.get<by_addr>(), std::move(peers.anchor)); + m_allow_local_ip = allow_local_ip; + return true; + } + + void peerlist_manager::get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + copy_peers(pl_gray, m_peers_gray.get<by_addr>()); + copy_peers(pl_white, m_peers_white.get<by_addr>()); + } + + void peerlist_manager::get_peerlist(peerlist_types& peers) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + peers.white.reserve(peers.white.size() + m_peers_white.size()); + peers.gray.reserve(peers.gray.size() + m_peers_gray.size()); + peers.anchor.reserve(peers.anchor.size() + m_peers_anchor.size()); + + copy_peers(peers.white, m_peers_white.get<by_addr>()); + copy_peers(peers.gray, m_peers_gray.get<by_addr>()); + copy_peers(peers.anchor, m_peers_anchor.get<by_addr>()); + } +} + +BOOST_CLASS_VERSION(nodetool::peerlist_types, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); +BOOST_CLASS_VERSION(nodetool::peerlist_join, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); + diff --git a/src/p2p/net_peerlist.h b/src/p2p/net_peerlist.h index 685fdc193..46726d7d8 100644 --- a/src/p2p/net_peerlist.h +++ b/src/p2p/net_peerlist.h @@ -30,33 +30,67 @@ #pragma once +#include <iosfwd> #include <list> -#include <set> -#include <map> -#include <boost/archive/binary_iarchive.hpp> -#include <boost/archive/portable_binary_oarchive.hpp> -#include <boost/archive/portable_binary_iarchive.hpp> -#include <boost/serialization/version.hpp> +#include <string> +#include <vector> #include <boost/multi_index_container.hpp> #include <boost/multi_index/ordered_index.hpp> #include <boost/multi_index/identity.hpp> #include <boost/multi_index/member.hpp> +#include <boost/optional/optional.hpp> #include <boost/range/adaptor/reversed.hpp> -#include "syncobj.h" +#include "cryptonote_config.h" +#include "net/enums.h" #include "net/local_ip.h" #include "p2p_protocol_defs.h" -#include "cryptonote_config.h" -#include "net_peerlist_boost_serialization.h" - - -#define CURRENT_PEERLIST_STORAGE_ARCHIVE_VER 6 +#include "syncobj.h" namespace nodetool { + struct peerlist_types + { + std::vector<peerlist_entry> white; + std::vector<peerlist_entry> gray; + std::vector<anchor_peerlist_entry> anchor; + }; + + class peerlist_storage + { + public: + peerlist_storage() + : m_types{} + {} + + //! \return Peers stored in stream `src` in `new_format` (portable archive or older non-portable). + static boost::optional<peerlist_storage> open(std::istream& src, const bool new_format); + + //! \return Peers stored in file at `path` + static boost::optional<peerlist_storage> open(const std::string& path); + + peerlist_storage(peerlist_storage&&) = default; + peerlist_storage(const peerlist_storage&) = delete; + + ~peerlist_storage() noexcept; + peerlist_storage& operator=(peerlist_storage&&) = default; + peerlist_storage& operator=(const peerlist_storage&) = delete; + + //! Save peers from `this` and `other` in stream `dest`. + bool store(std::ostream& dest, const peerlist_types& other) const; + + //! Save peers from `this` and `other` in one file at `path`. + bool store(const std::string& path, const peerlist_types& other) const; + + //! \return Peers in `zone` and from remove from `this`. + peerlist_types take_zone(epee::net_utils::zone zone); + + private: + peerlist_types m_types; + }; /************************************************************************/ /* */ @@ -64,13 +98,13 @@ namespace nodetool class peerlist_manager { public: - bool init(bool allow_local_ip); - bool deinit(); + bool init(peerlist_types&& peers, bool allow_local_ip); size_t get_white_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_white.size();} size_t get_gray_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_gray.size();} bool merge_peerlist(const std::vector<peerlist_entry>& outer_bs); bool get_peerlist_head(std::vector<peerlist_entry>& bs_head, uint32_t depth = P2P_DEFAULT_PEERS_IN_HANDSHAKE); - bool get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(peerlist_types& peers); bool get_white_peer_by_index(peerlist_entry& p, size_t i); bool get_gray_peer_by_index(peerlist_entry& p, size_t i); template<typename F> bool foreach(bool white, const F &f); @@ -136,18 +170,6 @@ namespace nodetool > peers_indexed; typedef boost::multi_index_container< - peerlist_entry, - boost::multi_index::indexed_by< - // access by peerlist_entry::id< - boost::multi_index::ordered_unique<boost::multi_index::tag<by_id>, boost::multi_index::member<peerlist_entry,uint64_t,&peerlist_entry::id> >, - // access by peerlist_entry::net_adress - boost::multi_index::ordered_unique<boost::multi_index::tag<by_addr>, boost::multi_index::member<peerlist_entry,epee::net_utils::network_address,&peerlist_entry::adr> >, - // sort by peerlist_entry::last_seen< - boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<peerlist_entry,int64_t,&peerlist_entry::last_seen> > - > - > peers_indexed_old; - - typedef boost::multi_index_container< anchor_peerlist_entry, boost::multi_index::indexed_by< // access by anchor_peerlist_entry::net_adress @@ -156,56 +178,8 @@ namespace nodetool boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<anchor_peerlist_entry,int64_t,&anchor_peerlist_entry::first_seen> > > > anchor_peers_indexed; - public: - - template <class Archive, class List, class Element, class t_version_type> - void serialize_peers(Archive &a, List &list, Element ple, const t_version_type ver) - { - if (typename Archive::is_saving()) - { - uint64_t size = list.size(); - a & size; - for (auto p: list) - { - a & p; - } - } - else - { - uint64_t size; - a & size; - list.clear(); - while (size--) - { - a & ple; - list.insert(ple); - } - } - } - - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - // at v6, we drop existing peerlists, because annoying change - if (ver < 6) - return; - - CRITICAL_REGION_LOCAL(m_peerlist_lock); - -#if 0 - // trouble loading more than one peer, can't find why - a & m_peers_white; - a & m_peers_gray; - a & m_peers_anchor; -#else - serialize_peers(a, m_peers_white, peerlist_entry(), ver); - serialize_peers(a, m_peers_gray, peerlist_entry(), ver); - serialize_peers(a, m_peers_anchor, anchor_peerlist_entry(), ver); -#endif - } private: - bool peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi); void trim_white_peerlist(); void trim_gray_peerlist(); @@ -220,34 +194,6 @@ namespace nodetool anchor_peers_indexed m_peers_anchor; }; //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::init(bool allow_local_ip) - { - m_allow_local_ip = allow_local_ip; - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::deinit() - { - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi) - { - for(auto x: pio) - { - auto by_addr_it = pi.get<by_addr>().find(x.adr); - if(by_addr_it == pi.get<by_addr>().end()) - { - pi.insert(x); - } - } - - return true; - } - //-------------------------------------------------------------------------------------------------- inline void peerlist_manager::trim_gray_peerlist() { while(m_peers_gray.size() > P2P_LOCAL_GRAY_PEERLIST_LIMIT) @@ -337,27 +283,6 @@ namespace nodetool return true; } //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) - { - CRITICAL_REGION_LOCAL(m_peerlist_lock); - peers_indexed::index<by_time>::type& by_time_index_gr=m_peers_gray.get<by_time>(); - pl_gray.resize(pl_gray.size() + by_time_index_gr.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_gr)) - { - pl_gray.push_back(vl); - } - - peers_indexed::index<by_time>::type& by_time_index_wt=m_peers_white.get<by_time>(); - pl_white.resize(pl_white.size() + by_time_index_wt.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_wt)) - { - pl_white.push_back(vl); - } - - return true; - } - //-------------------------------------------------------------------------------------------------- template<typename F> inline bool peerlist_manager::foreach(bool white, const F &f) { @@ -559,4 +484,3 @@ namespace nodetool //-------------------------------------------------------------------------------------------------- } -BOOST_CLASS_VERSION(nodetool::peerlist_manager, CURRENT_PEERLIST_STORAGE_ARCHIVE_VER) diff --git a/src/p2p/net_peerlist_boost_serialization.h b/src/p2p/net_peerlist_boost_serialization.h index 2f4a7e661..6c891581f 100644 --- a/src/p2p/net_peerlist_boost_serialization.h +++ b/src/p2p/net_peerlist_boost_serialization.h @@ -30,7 +30,12 @@ #pragma once +#include <cstring> + +#include "common/expect.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" +#include "net/i2p_address.h" #include "p2p/p2p_protocol_defs.h" #ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED @@ -42,24 +47,40 @@ namespace boost namespace serialization { template <class T, class Archive> - inline void do_serialize(Archive &a, epee::net_utils::network_address& na, T local) + inline void do_serialize(boost::mpl::false_, Archive &a, epee::net_utils::network_address& na) + { + T addr{}; + a & addr; + na = std::move(addr); + } + + template <class T, class Archive> + inline void do_serialize(boost::mpl::true_, Archive &a, const epee::net_utils::network_address& na) { - if (typename Archive::is_saving()) local = na.as<T>(); - a & local; - if (!typename Archive::is_saving()) na = local; + a & na.as<T>(); } + template <class Archive, class ver_type> inline void serialize(Archive &a, epee::net_utils::network_address& na, const ver_type ver) { + static constexpr const typename Archive::is_saving is_saving{}; + uint8_t type; - if (typename Archive::is_saving()) - type = na.get_type_id(); + if (is_saving) + type = uint8_t(na.get_type_id()); a & type; - switch (type) + switch (epee::net_utils::address_type(type)) { - case epee::net_utils::ipv4_network_address::ID: - do_serialize(a, na, epee::net_utils::ipv4_network_address{0, 0}); - break; + case epee::net_utils::ipv4_network_address::get_type_id(): + do_serialize<epee::net_utils::ipv4_network_address>(is_saving, a, na); + break; + case net::tor_address::get_type_id(): + do_serialize<net::tor_address>(is_saving, a, na); + break; + case net::i2p_address::get_type_id(): + do_serialize<net::i2p_address>(is_saving, a, na); + break; + case epee::net_utils::address_type::invalid: default: throw std::runtime_error("Unsupported network address type"); } @@ -76,6 +97,88 @@ namespace boost } template <class Archive, class ver_type> + inline void save(Archive& a, const net::tor_address& na, const ver_type) + { + const size_t length = std::strlen(na.host_str()); + if (length > 255) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + const uint16_t port{na.port()}; + const uint8_t len = length; + a & port; + a & len; + a.save_binary(na.host_str(), length); + } + + template <class Archive, class ver_type> + inline void save(Archive& a, const net::i2p_address& na, const ver_type) + { + const size_t length = std::strlen(na.host_str()); + if (length > 255) + MONERO_THROW(net::error::invalid_i2p_address, "i2p address too long"); + + const uint16_t port{na.port()}; + const uint8_t len = length; + a & port; + a & len; + a.save_binary(na.host_str(), length); + } + + template <class Archive, class ver_type> + inline void load(Archive& a, net::tor_address& na, const ver_type) + { + uint16_t port = 0; + uint8_t length = 0; + a & port; + a & length; + + if (length > net::tor_address::buffer_size()) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + char host[net::tor_address::buffer_size()] = {0}; + a.load_binary(host, length); + host[sizeof(host) - 1] = 0; + + if (std::strcmp(host, net::tor_address::unknown_str()) == 0) + na = net::tor_address::unknown(); + else + na = MONERO_UNWRAP(net::tor_address::make(host, port)); + } + + template <class Archive, class ver_type> + inline void load(Archive& a, net::i2p_address& na, const ver_type) + { + uint16_t port = 0; + uint8_t length = 0; + a & port; + a & length; + + if (length > net::i2p_address::buffer_size()) + MONERO_THROW(net::error::invalid_i2p_address, "i2p address too long"); + + char host[net::i2p_address::buffer_size()] = {0}; + a.load_binary(host, length); + host[sizeof(host) - 1] = 0; + + if (std::strcmp(host, net::i2p_address::unknown_str()) == 0) + na = net::i2p_address::unknown(); + else + na = MONERO_UNWRAP(net::i2p_address::make(host, port)); + } + + template <class Archive, class ver_type> + inline void serialize(Archive &a, net::tor_address& na, const ver_type ver) + { + boost::serialization::split_free(a, na, ver); + } + + template <class Archive, class ver_type> + inline void serialize(Archive &a, net::i2p_address& na, const ver_type ver) + { + boost::serialization::split_free(a, na, ver); + } + + template <class Archive, class ver_type> inline void serialize(Archive &a, nodetool::peerlist_entry& pl, const ver_type ver) { a & pl.adr; diff --git a/src/p2p/p2p_protocol_defs.h b/src/p2p/p2p_protocol_defs.h index 939cedf42..e9449b950 100644 --- a/src/p2p/p2p_protocol_defs.h +++ b/src/p2p/p2p_protocol_defs.h @@ -34,6 +34,8 @@ #include <boost/serialization/version.hpp> #include "serialization/keyvalue_serialization.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" // needed for serialization +#include "net/i2p_address.h" // needed for serialization #include "misc_language.h" #include "string_tools.h" #include "time_helper.h" @@ -204,7 +206,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); @@ -263,7 +265,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); diff --git a/src/ringct/bulletproofs.cc b/src/ringct/bulletproofs.cc index d485fb748..b5fd626dc 100644 --- a/src/ringct/bulletproofs.cc +++ b/src/ringct/bulletproofs.cc @@ -202,20 +202,36 @@ static rct::keyV vector_powers(const rct::key &x, size_t n) } /* Given a scalar, return the sum of its powers from 0 to n-1 */ -static rct::key vector_power_sum(const rct::key &x, size_t n) +static rct::key vector_power_sum(rct::key x, size_t n) { if (n == 0) return rct::zero(); rct::key res = rct::identity(); if (n == 1) return res; - rct::key prev = x; - for (size_t i = 1; i < n; ++i) + + const bool is_power_of_2 = (n & (n - 1)) == 0; + if (is_power_of_2) { - if (i > 1) - sc_mul(prev.bytes, prev.bytes, x.bytes); - sc_add(res.bytes, res.bytes, prev.bytes); + sc_add(res.bytes, res.bytes, x.bytes); + while (n > 2) + { + sc_mul(x.bytes, x.bytes, x.bytes); + sc_muladd(res.bytes, x.bytes, res.bytes, res.bytes); + n /= 2; + } + } + else + { + rct::key prev = x; + for (size_t i = 1; i < n; ++i) + { + if (i > 1) + sc_mul(prev.bytes, prev.bytes, x.bytes); + sc_add(res.bytes, res.bytes, prev.bytes); + } } + return res; } diff --git a/src/rpc/CMakeLists.txt b/src/rpc/CMakeLists.txt index d2c4a33cb..60cae036c 100644 --- a/src/rpc/CMakeLists.txt +++ b/src/rpc/CMakeLists.txt @@ -112,6 +112,7 @@ target_link_libraries(rpc common cryptonote_core cryptonote_protocol + net version ${Boost_REGEX_LIBRARY} ${Boost_THREAD_LIBRARY} diff --git a/src/rpc/core_rpc_server.cpp b/src/rpc/core_rpc_server.cpp index d308dd63d..d18774149 100644 --- a/src/rpc/core_rpc_server.cpp +++ b/src/rpc/core_rpc_server.cpp @@ -42,6 +42,7 @@ using namespace epee; #include "cryptonote_basic/account.h" #include "cryptonote_basic/cryptonote_basic_impl.h" #include "misc_language.h" +#include "net/parse.h" #include "storages/http_abstract_invoke.h" #include "crypto/hash.h" #include "rpc/rpc_args.h" @@ -75,6 +76,11 @@ namespace cryptonote command_line::add_arg(desc, arg_rpc_bind_port); command_line::add_arg(desc, arg_rpc_restricted_bind_port); command_line::add_arg(desc, arg_restricted_rpc); + command_line::add_arg(desc, arg_rpc_ssl); + command_line::add_arg(desc, arg_rpc_ssl_private_key); + command_line::add_arg(desc, arg_rpc_ssl_certificate); + command_line::add_arg(desc, arg_rpc_ssl_allowed_certificates); + command_line::add_arg(desc, arg_rpc_ssl_allow_any_cert); command_line::add_arg(desc, arg_bootstrap_daemon_address); command_line::add_arg(desc, arg_bootstrap_daemon_login); cryptonote::rpc_args::init_options(desc); @@ -111,11 +117,11 @@ namespace cryptonote epee::net_utils::http::login login; login.username = bootstrap_daemon_login.substr(0, loc); login.password = bootstrap_daemon_login.substr(loc + 1); - m_http_client.set_server(m_bootstrap_daemon_address, login, false); + m_http_client.set_server(m_bootstrap_daemon_address, login, epee::net_utils::ssl_support_t::e_ssl_support_autodetect); } else { - m_http_client.set_server(m_bootstrap_daemon_address, boost::none, false); + m_http_client.set_server(m_bootstrap_daemon_address, boost::none, epee::net_utils::ssl_support_t::e_ssl_support_autodetect); } m_should_use_bootstrap_daemon = true; } @@ -130,9 +136,32 @@ namespace cryptonote if (rpc_config->login) http_login.emplace(std::move(rpc_config->login->username), std::move(rpc_config->login->password).password()); + epee::net_utils::ssl_support_t ssl_support; + const std::string ssl = command_line::get_arg(vm, arg_rpc_ssl); + if (!epee::net_utils::ssl_support_from_string(ssl_support, ssl)) + { + MFATAL("Invalid RPC SSL support: " << ssl); + return false; + } + const std::string ssl_private_key = command_line::get_arg(vm, arg_rpc_ssl_private_key); + const std::string ssl_certificate = command_line::get_arg(vm, arg_rpc_ssl_certificate); + const std::vector<std::string> ssl_allowed_certificate_paths = command_line::get_arg(vm, arg_rpc_ssl_allowed_certificates); + std::list<std::string> ssl_allowed_certificates; + for (const std::string &path: ssl_allowed_certificate_paths) + { + ssl_allowed_certificates.push_back({}); + if (!epee::file_io_utils::load_file_to_string(path, ssl_allowed_certificates.back())) + { + MERROR("Failed to load certificate: " << path); + ssl_allowed_certificates.back() = std::string(); + } + } + const bool ssl_allow_any_cert = command_line::get_arg(vm, arg_rpc_ssl_allow_any_cert); + auto rng = [](size_t len, uint8_t *ptr){ return crypto::rand(len, ptr); }; return epee::http_server_impl_base<core_rpc_server, connection_context>::init( - rng, std::move(port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login) + rng, std::move(port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login), + ssl_support, std::make_pair(ssl_private_key, ssl_certificate), ssl_allowed_certificates, ssl_allow_any_cert ); } //------------------------------------------------------------------------------------------------------------------------------ @@ -185,12 +214,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -902,12 +931,12 @@ namespace cryptonote PERF_TIMER(on_get_peer_list); std::vector<nodetool::peerlist_entry> white_list; std::vector<nodetool::peerlist_entry> gray_list; - m_p2p.get_peerlist_manager().get_peerlist_full(gray_list, white_list); + m_p2p.get_public_peerlist(gray_list, white_list); res.white_list.reserve(white_list.size()); for (auto & entry : white_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.white_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -917,7 +946,7 @@ namespace cryptonote res.gray_list.reserve(gray_list.size()); for (auto & entry : gray_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.gray_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -1321,6 +1350,7 @@ namespace cryptonote response.block_size = response.block_weight = m_core.get_blockchain_storage().get_db().get_block_weight(height); response.num_txes = blk.tx_hashes.size(); response.pow_hash = fill_pow_hash ? string_tools::pod_to_hex(get_block_longhash(blk, height)) : ""; + response.long_term_weight = m_core.get_blockchain_storage().get_db().get_block_long_term_weight(height); return true; } //------------------------------------------------------------------------------------------------------------------------------ @@ -1646,12 +1676,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -1730,12 +1760,14 @@ namespace cryptonote epee::net_utils::network_address na; if (!i->host.empty()) { - if (!epee::net_utils::create_network_address(na, i->host)) + auto na_parsed = net::get_network_address(i->host, 0); + if (!na_parsed) { error_resp.code = CORE_RPC_ERROR_CODE_WRONG_PARAM; error_resp.message = "Unsupported host type"; return false; } + na = std::move(*na_parsed); } else { @@ -1958,11 +1990,7 @@ namespace cryptonote bool core_rpc_server::on_out_peers(const COMMAND_RPC_OUT_PEERS::request& req, COMMAND_RPC_OUT_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_out_peers); - size_t n_connections = m_p2p.get_outgoing_connections_count(); - size_t n_delete = (n_connections > req.out_peers) ? n_connections - req.out_peers : 0; - m_p2p.m_config.m_net_config.max_out_connection_count = req.out_peers; - if (n_delete) - m_p2p.delete_out_connections(n_delete); + m_p2p.change_max_out_public_peers(req.out_peers); res.status = CORE_RPC_STATUS_OK; return true; } @@ -1970,11 +1998,7 @@ namespace cryptonote bool core_rpc_server::on_in_peers(const COMMAND_RPC_IN_PEERS::request& req, COMMAND_RPC_IN_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_in_peers); - size_t n_connections = m_p2p.get_incoming_connections_count(); - size_t n_delete = (n_connections > req.in_peers) ? n_connections - req.in_peers : 0; - m_p2p.m_config.m_net_config.max_in_connection_count = req.in_peers; - if (n_delete) - m_p2p.delete_in_connections(n_delete); + m_p2p.change_max_in_public_peers(req.in_peers); res.status = CORE_RPC_STATUS_OK; return true; } @@ -2322,6 +2346,35 @@ namespace cryptonote , false }; + const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl = { + "rpc-ssl" + , "Enable SSL on RPC connections: enabled|disabled|autodetect" + , "autodetect" + }; + + const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl_private_key = { + "rpc-ssl-private-key" + , "Path to a PEM format private key" + , "" + }; + + const command_line::arg_descriptor<std::string> core_rpc_server::arg_rpc_ssl_certificate = { + "rpc-ssl-certificate" + , "Path to a PEM format certificate" + , "" + }; + + const command_line::arg_descriptor<std::vector<std::string>> core_rpc_server::arg_rpc_ssl_allowed_certificates = { + "rpc-ssl-allowed-certificates" + , "List of paths to PEM format certificates of allowed peers (all allowed if empty)" + }; + + const command_line::arg_descriptor<bool> core_rpc_server::arg_rpc_ssl_allow_any_cert = { + "rpc-ssl-allow-any-cert" + , "Allow any peer certificate, rather than just those on the allowed list" + , false + }; + const command_line::arg_descriptor<std::string> core_rpc_server::arg_bootstrap_daemon_address = { "bootstrap-daemon-address" , "URL of a 'bootstrap' remote daemon that the connected wallets can use while this daemon is still not fully synced" diff --git a/src/rpc/core_rpc_server.h b/src/rpc/core_rpc_server.h index 62a377841..da1907af2 100644 --- a/src/rpc/core_rpc_server.h +++ b/src/rpc/core_rpc_server.h @@ -56,6 +56,11 @@ namespace cryptonote static const command_line::arg_descriptor<std::string, false, true, 2> arg_rpc_bind_port; static const command_line::arg_descriptor<std::string> arg_rpc_restricted_bind_port; static const command_line::arg_descriptor<bool> arg_restricted_rpc; + static const command_line::arg_descriptor<std::string> arg_rpc_ssl; + static const command_line::arg_descriptor<std::string> arg_rpc_ssl_private_key; + static const command_line::arg_descriptor<std::string> arg_rpc_ssl_certificate; + static const command_line::arg_descriptor<std::vector<std::string>> arg_rpc_ssl_allowed_certificates; + static const command_line::arg_descriptor<bool> arg_rpc_ssl_allow_any_cert; static const command_line::arg_descriptor<std::string> arg_bootstrap_daemon_address; static const command_line::arg_descriptor<std::string> arg_bootstrap_daemon_login; diff --git a/src/rpc/core_rpc_server_commands_defs.h b/src/rpc/core_rpc_server_commands_defs.h index dfad5d6a7..e52e4fc67 100644 --- a/src/rpc/core_rpc_server_commands_defs.h +++ b/src/rpc/core_rpc_server_commands_defs.h @@ -1172,6 +1172,7 @@ namespace cryptonote uint64_t block_weight; uint64_t num_txes; std::string pow_hash; + uint64_t long_term_weight; BEGIN_KV_SERIALIZE_MAP() KV_SERIALIZE(major_version) @@ -1190,6 +1191,7 @@ namespace cryptonote KV_SERIALIZE_OPT(block_weight, (uint64_t)0) KV_SERIALIZE(num_txes) KV_SERIALIZE(pow_hash) + KV_SERIALIZE_OPT(long_term_weight, (uint64_t)0) END_KV_SERIALIZE_MAP() }; diff --git a/src/rpc/daemon_handler.cpp b/src/rpc/daemon_handler.cpp index e2885dbb5..871f7d368 100644 --- a/src/rpc/daemon_handler.cpp +++ b/src/rpc/daemon_handler.cpp @@ -423,13 +423,13 @@ namespace rpc res.info.alt_blocks_count = chain.get_alternative_blocks_count(); - uint64_t total_conn = m_p2p.get_connections_count(); - res.info.outgoing_connections_count = m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = m_p2p.get_public_connections_count(); + res.info.outgoing_connections_count = m_p2p.get_public_outgoing_connections_count(); res.info.incoming_connections_count = total_conn - res.info.outgoing_connections_count; - res.info.white_peerlist_size = m_p2p.get_peerlist_manager().get_white_peers_count(); + res.info.white_peerlist_size = m_p2p.get_public_white_peers_count(); - res.info.grey_peerlist_size = m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.info.grey_peerlist_size = m_p2p.get_public_gray_peers_count(); res.info.mainnet = m_core.get_nettype() == MAINNET; res.info.testnet = m_core.get_nettype() == TESTNET; diff --git a/src/serialization/binary_archive.h b/src/serialization/binary_archive.h index f47a4494d..242b8fd68 100644 --- a/src/serialization/binary_archive.h +++ b/src/serialization/binary_archive.h @@ -99,7 +99,7 @@ struct binary_archive<false> : public binary_archive_base<std::istream, false> { explicit binary_archive(stream_type &s) : base_type(s) { - stream_type::streampos pos = stream_.tellg(); + stream_type::pos_type pos = stream_.tellg(); stream_.seekg(0, std::ios_base::end); eof_pos_ = stream_.tellg(); stream_.seekg(pos); diff --git a/src/simplewallet/simplewallet.cpp b/src/simplewallet/simplewallet.cpp index 3cbfb760b..ceb844fbf 100644 --- a/src/simplewallet/simplewallet.cpp +++ b/src/simplewallet/simplewallet.cpp @@ -116,7 +116,7 @@ typedef cryptonote::simple_wallet sw; #define LONG_PAYMENT_ID_SUPPORT_CHECK() \ do { \ if (!m_long_payment_id_support) { \ - fail_msg_writer() << tr("Long payment IDs are obsolete. Use --long-payment-id-support if you really must use one."); \ + fail_msg_writer() << tr("Long payment IDs are obsolete. Use --long-payment-id-support if you really must use one, and warn the recipient they are using an obsolete feature that will disappear in the future."); \ return true; \ } \ } while(0) @@ -8402,7 +8402,8 @@ bool simple_wallet::status(const std::vector<std::string> &args) { uint64_t local_height = m_wallet->get_blockchain_current_height(); uint32_t version = 0; - if (!m_wallet->check_connection(&version)) + bool ssl = false; + if (!m_wallet->check_connection(&version, &ssl)) { success_msg_writer() << "Refreshed " << local_height << "/?, no daemon connected"; return true; @@ -8414,7 +8415,7 @@ bool simple_wallet::status(const std::vector<std::string> &args) { bool synced = local_height == bc_height; success_msg_writer() << "Refreshed " << local_height << "/" << bc_height << ", " << (synced ? "synced" : "syncing") - << ", daemon RPC v" << get_version_string(version); + << ", daemon RPC v" << get_version_string(version) << ", " << (ssl ? "SSL" : "no SSL"); } else { diff --git a/src/wallet/api/wallet.cpp b/src/wallet/api/wallet.cpp index 935a8d51c..2b7853330 100644 --- a/src/wallet/api/wallet.cpp +++ b/src/wallet/api/wallet.cpp @@ -1924,7 +1924,7 @@ bool WalletImpl::verifyMessageWithPublicKey(const std::string &message, const st bool WalletImpl::connectToDaemon() { - bool result = m_wallet->check_connection(NULL, DEFAULT_CONNECTION_TIMEOUT_MILLIS); + bool result = m_wallet->check_connection(NULL, NULL, DEFAULT_CONNECTION_TIMEOUT_MILLIS); if (!result) { setStatusError("Error connecting to daemon at " + m_wallet->get_daemon_address()); } else { @@ -1937,7 +1937,7 @@ bool WalletImpl::connectToDaemon() Wallet::ConnectionStatus WalletImpl::connected() const { uint32_t version = 0; - m_is_connected = m_wallet->check_connection(&version, DEFAULT_CONNECTION_TIMEOUT_MILLIS); + m_is_connected = m_wallet->check_connection(&version, NULL, DEFAULT_CONNECTION_TIMEOUT_MILLIS); if (!m_is_connected) return Wallet::ConnectionStatus_Disconnected; // Version check is not implemented in light wallets nodes/wallets diff --git a/src/wallet/wallet2.cpp b/src/wallet/wallet2.cpp index c02d10ab4..48e794616 100644 --- a/src/wallet/wallet2.cpp +++ b/src/wallet/wallet2.cpp @@ -237,6 +237,11 @@ struct options { const command_line::arg_descriptor<std::string> password_file = {"password-file", tools::wallet2::tr("Wallet password file"), "", true}; const command_line::arg_descriptor<int> daemon_port = {"daemon-port", tools::wallet2::tr("Use daemon instance at port <arg> instead of 18081"), 0}; const command_line::arg_descriptor<std::string> daemon_login = {"daemon-login", tools::wallet2::tr("Specify username[:password] for daemon RPC client"), "", true}; + const command_line::arg_descriptor<std::string> daemon_ssl = {"daemon-ssl", tools::wallet2::tr("Enable SSL on daemon RPC connections: enabled|disabled|autodetect"), "autodetect"}; + const command_line::arg_descriptor<std::string> daemon_ssl_private_key = {"daemon-ssl-private-key", tools::wallet2::tr("Path to a PEM format private key"), ""}; + const command_line::arg_descriptor<std::string> daemon_ssl_certificate = {"daemon-ssl-certificate", tools::wallet2::tr("Path to a PEM format certificate"), ""}; + const command_line::arg_descriptor<std::vector<std::string>> daemon_ssl_allowed_certificates = {"daemon-ssl-allowed-certificates", tools::wallet2::tr("List of paths to PEM format certificates of allowed RPC servers")}; + const command_line::arg_descriptor<bool> daemon_ssl_allow_any_cert = {"daemon-ssl-allow-any-cert", tools::wallet2::tr("Allow any SSL certificate from the daemon"), false}; const command_line::arg_descriptor<bool> testnet = {"testnet", tools::wallet2::tr("For testnet. Daemon must also be launched with --testnet flag"), false}; const command_line::arg_descriptor<bool> stagenet = {"stagenet", tools::wallet2::tr("For stagenet. Daemon must also be launched with --stagenet flag"), false}; const command_line::arg_descriptor<std::string, false, true, 2> shared_ringdb_dir = { @@ -308,6 +313,14 @@ std::unique_ptr<tools::wallet2> make_basic(const boost::program_options::variabl auto daemon_port = command_line::get_arg(vm, opts.daemon_port); auto device_name = command_line::get_arg(vm, opts.hw_device); auto device_derivation_path = command_line::get_arg(vm, opts.hw_device_derivation_path); + auto daemon_ssl_private_key = command_line::get_arg(vm, opts.daemon_ssl_private_key); + auto daemon_ssl_certificate = command_line::get_arg(vm, opts.daemon_ssl_certificate); + auto daemon_ssl_allowed_certificates = command_line::get_arg(vm, opts.daemon_ssl_allowed_certificates); + auto daemon_ssl_allow_any_cert = command_line::get_arg(vm, opts.daemon_ssl_allow_any_cert); + auto daemon_ssl = command_line::get_arg(vm, opts.daemon_ssl); + epee::net_utils::ssl_support_t ssl_support; + THROW_WALLET_EXCEPTION_IF(!epee::net_utils::ssl_support_from_string(ssl_support, daemon_ssl), tools::error::wallet_internal_error, + tools::wallet2::tr("Invalid argument for ") + std::string(opts.daemon_ssl.name)); THROW_WALLET_EXCEPTION_IF(!daemon_address.empty() && !daemon_host.empty() && 0 != daemon_port, tools::error::wallet_internal_error, tools::wallet2::tr("can't specify daemon host or port more than once")); @@ -358,8 +371,20 @@ std::unique_ptr<tools::wallet2> make_basic(const boost::program_options::variabl catch (const std::exception &e) { } } + std::list<std::string> ssl_allowed_certificates; + for (const std::string &path: daemon_ssl_allowed_certificates) + { + ssl_allowed_certificates.push_back({}); + if (!epee::file_io_utils::load_file_to_string(path, ssl_allowed_certificates.back())) + { + MERROR("Failed to load certificate: " << path); + ssl_allowed_certificates.back() = std::string(); + } + } + std::unique_ptr<tools::wallet2> wallet(new tools::wallet2(nettype, kdf_rounds, unattended)); - wallet->init(std::move(daemon_address), std::move(login), 0, false, *trusted_daemon); + wallet->init(std::move(daemon_address), std::move(login), 0, *trusted_daemon, ssl_support, std::make_pair(daemon_ssl_private_key, daemon_ssl_certificate), ssl_allowed_certificates, daemon_ssl_allow_any_cert); + boost::filesystem::path ringdb_path = command_line::get_arg(vm, opts.shared_ringdb_dir); wallet->set_ring_database(ringdb_path.string()); wallet->get_message_store().set_options(vm); @@ -373,7 +398,7 @@ std::unique_ptr<tools::wallet2> make_basic(const boost::program_options::variabl } catch (const std::exception &e) { - MERROR("Failed to parse tx notify spec"); + MERROR("Failed to parse tx notify spec: " << e.what()); } return wallet; @@ -867,7 +892,7 @@ wallet_keys_unlocker::wallet_keys_unlocker(wallet2 &w, const boost::optional<too w(w), locked(password != boost::none) { - if (!locked || w.is_unattended() || w.ask_password() != tools::wallet2::AskPasswordToDecrypt) + if (!locked || w.is_unattended() || w.ask_password() != tools::wallet2::AskPasswordToDecrypt || w.watch_only()) { locked = false; return; @@ -1015,6 +1040,11 @@ void wallet2::init_options(boost::program_options::options_description& desc_par command_line::add_arg(desc_params, opts.password_file); command_line::add_arg(desc_params, opts.daemon_port); command_line::add_arg(desc_params, opts.daemon_login); + command_line::add_arg(desc_params, opts.daemon_ssl); + command_line::add_arg(desc_params, opts.daemon_ssl_private_key); + command_line::add_arg(desc_params, opts.daemon_ssl_certificate); + command_line::add_arg(desc_params, opts.daemon_ssl_allowed_certificates); + command_line::add_arg(desc_params, opts.daemon_ssl_allow_any_cert); command_line::add_arg(desc_params, opts.testnet); command_line::add_arg(desc_params, opts.stagenet); command_line::add_arg(desc_params, opts.shared_ringdb_dir); @@ -1066,7 +1096,7 @@ std::unique_ptr<wallet2> wallet2::make_dummy(const boost::program_options::varia } //---------------------------------------------------------------------------------------------------- -bool wallet2::init(std::string daemon_address, boost::optional<epee::net_utils::http::login> daemon_login, uint64_t upper_transaction_weight_limit, bool ssl, bool trusted_daemon) +bool wallet2::init(std::string daemon_address, boost::optional<epee::net_utils::http::login> daemon_login, uint64_t upper_transaction_weight_limit, bool trusted_daemon, epee::net_utils::ssl_support_t ssl_support, const std::pair<std::string, std::string> &private_key_and_certificate_path, const std::list<std::string> &allowed_certificates, bool allow_any_cert) { m_checkpoints.init_default_checkpoints(m_nettype); if(m_http_client.is_connected()) @@ -1076,8 +1106,7 @@ bool wallet2::init(std::string daemon_address, boost::optional<epee::net_utils:: m_daemon_address = std::move(daemon_address); m_daemon_login = std::move(daemon_login); m_trusted_daemon = trusted_daemon; - // When switching from light wallet to full wallet, we need to reset the height we got from lw node. - return m_http_client.set_server(get_daemon_address(), get_daemon_login(), ssl); + return m_http_client.set_server(get_daemon_address(), get_daemon_login(), ssl_support, private_key_and_certificate_path, allowed_certificates, allow_any_cert); } //---------------------------------------------------------------------------------------------------- bool wallet2::is_deterministic() const @@ -1426,7 +1455,7 @@ static uint64_t decodeRct(const rct::rctSig & rv, const crypto::key_derivation & } } //---------------------------------------------------------------------------------------------------- -void wallet2::scan_output(const cryptonote::transaction &tx, const crypto::public_key &tx_pub_key, size_t i, tx_scan_info_t &tx_scan_info, int &num_vouts_received, std::unordered_map<cryptonote::subaddress_index, uint64_t> &tx_money_got_in_outs, std::vector<size_t> &outs, bool pool) +void wallet2::scan_output(const cryptonote::transaction &tx, bool miner_tx, const crypto::public_key &tx_pub_key, size_t i, tx_scan_info_t &tx_scan_info, int &num_vouts_received, std::unordered_map<cryptonote::subaddress_index, uint64_t> &tx_money_got_in_outs, std::vector<size_t> &outs, bool pool) { THROW_WALLET_EXCEPTION_IF(i >= tx.vout.size(), error::wallet_internal_error, "Invalid vout index"); @@ -1459,11 +1488,14 @@ void wallet2::scan_output(const cryptonote::transaction &tx, const crypto::publi error::wallet_internal_error, "key_image generated ephemeral public key not matched with output_key"); } + THROW_WALLET_EXCEPTION_IF(std::find(outs.begin(), outs.end(), i) != outs.end(), error::wallet_internal_error, "Same output cannot be added twice"); outs.push_back(i); - if (tx_scan_info.money_transfered == 0) + if (tx_scan_info.money_transfered == 0 && !miner_tx) { tx_scan_info.money_transfered = tools::decodeRct(tx.rct_signatures, tx_scan_info.received->derivation, i, tx_scan_info.mask, m_account.get_device()); } + THROW_WALLET_EXCEPTION_IF(tx_money_got_in_outs[tx_scan_info.received->index] >= std::numeric_limits<uint64_t>::max() - tx_scan_info.money_transfered, + error::wallet_internal_error, "Overflow in received amounts"); tx_money_got_in_outs[tx_scan_info.received->index] += tx_scan_info.money_transfered; tx_scan_info.amount = tx_scan_info.money_transfered; ++num_vouts_received; @@ -1641,7 +1673,7 @@ void wallet2::process_new_transaction(const crypto::hash &txid, const cryptonote if (tx_scan_info[i].received) { hwdev.conceal_derivation(tx_scan_info[i].received->derivation, tx_pub_key, additional_tx_pub_keys.data, derivation, additional_derivations); - scan_output(tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); + scan_output(tx, miner_tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); } } } @@ -1664,7 +1696,7 @@ void wallet2::process_new_transaction(const crypto::hash &txid, const cryptonote if (tx_scan_info[i].received) { hwdev.conceal_derivation(tx_scan_info[i].received->derivation, tx_pub_key, additional_tx_pub_keys.data, derivation, additional_derivations); - scan_output(tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); + scan_output(tx, miner_tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); } } } @@ -1680,7 +1712,7 @@ void wallet2::process_new_transaction(const crypto::hash &txid, const cryptonote boost::unique_lock<hw::device> hwdev_lock (hwdev); hwdev.set_mode(hw::device::NONE); hwdev.conceal_derivation(tx_scan_info[i].received->derivation, tx_pub_key, additional_tx_pub_keys.data, derivation, additional_derivations); - scan_output(tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); + scan_output(tx, miner_tx, tx_pub_key, i, tx_scan_info[i], num_vouts_received, tx_money_got_in_outs, outs, pool); } } } @@ -2969,6 +3001,11 @@ void wallet2::refresh(bool trusted_daemon, uint64_t start_height, uint64_t & blo { LOG_PRINT_L1("Another try pull_blocks (try_count=" << try_count << ")..."); first = true; + start_height = 0; + blocks.clear(); + parsed_blocks.clear(); + short_chain_history.clear(); + get_short_chain_history(short_chain_history, 1); ++try_count; } else @@ -4848,7 +4885,7 @@ bool wallet2::prepare_file_names(const std::string& file_path) return true; } //---------------------------------------------------------------------------------------------------- -bool wallet2::check_connection(uint32_t *version, uint32_t timeout) +bool wallet2::check_connection(uint32_t *version, bool *ssl, uint32_t timeout) { THROW_WALLET_EXCEPTION_IF(!m_is_initialized, error::wallet_not_initialized); @@ -4856,15 +4893,20 @@ bool wallet2::check_connection(uint32_t *version, uint32_t timeout) // TODO: Add light wallet version check. if(m_light_wallet) { - version = 0; + if (version) + *version = 0; + if (ssl) + *ssl = m_light_wallet_connected; // light wallet is always SSL return m_light_wallet_connected; } - if(!m_http_client.is_connected()) + if(!m_http_client.is_connected(ssl)) { m_node_rpc_proxy.invalidate(); if (!m_http_client.connect(std::chrono::milliseconds(timeout))) return false; + if(!m_http_client.is_connected(ssl)) + return false; } if (version) @@ -11308,6 +11350,7 @@ uint64_t wallet2::import_key_images(const std::vector<std::pair<crypto::key_imag THROW_WALLET_EXCEPTION_IF(!r, error::wallet_internal_error, "Failed to generate key derivation"); } size_t output_index = 0; + bool miner_tx = cryptonote::is_coinbase(spent_tx); for (const cryptonote::tx_out& out : spent_tx.vout) { tx_scan_info_t tx_scan_info; @@ -11315,11 +11358,13 @@ uint64_t wallet2::import_key_images(const std::vector<std::pair<crypto::key_imag THROW_WALLET_EXCEPTION_IF(tx_scan_info.error, error::wallet_internal_error, "check_acc_out_precomp failed"); if (tx_scan_info.received) { - if (tx_scan_info.money_transfered == 0) + if (tx_scan_info.money_transfered == 0 && !miner_tx) { rct::key mask; tx_scan_info.money_transfered = tools::decodeRct(spent_tx.rct_signatures, tx_scan_info.received->derivation, output_index, mask, hwdev); } + THROW_WALLET_EXCEPTION_IF(tx_money_got_in_outs >= std::numeric_limits<uint64_t>::max() - tx_scan_info.money_transfered, + error::wallet_internal_error, "Overflow in received amounts"); tx_money_got_in_outs += tx_scan_info.money_transfered; } ++output_index; diff --git a/src/wallet/wallet2.h b/src/wallet/wallet2.h index 3b2dd6076..ea1172f40 100644 --- a/src/wallet/wallet2.h +++ b/src/wallet/wallet2.h @@ -676,7 +676,11 @@ namespace tools bool deinit(); bool init(std::string daemon_address = "http://localhost:8080", - boost::optional<epee::net_utils::http::login> daemon_login = boost::none, uint64_t upper_transaction_weight_limit = 0, bool ssl = false, bool trusted_daemon = false); + boost::optional<epee::net_utils::http::login> daemon_login = boost::none, uint64_t upper_transaction_weight_limit = 0, + bool trusted_daemon = true, + epee::net_utils::ssl_support_t ssl_support = epee::net_utils::ssl_support_t::e_ssl_support_autodetect, + const std::pair<std::string, std::string> &private_key_and_certificate_path = {}, + const std::list<std::string> &allowed_certificates = {}, bool allow_any_cert = false); void stop() { m_run.store(false, std::memory_order_relaxed); m_message_store.stop(); } @@ -800,7 +804,7 @@ namespace tools bool sign_multisig_tx_to_file(multisig_tx_set &exported_txs, const std::string &filename, std::vector<crypto::hash> &txids); std::vector<pending_tx> create_unmixable_sweep_transactions(); void discard_unmixable_outputs(); - bool check_connection(uint32_t *version = NULL, uint32_t timeout = 200000); + bool check_connection(uint32_t *version = NULL, bool *ssl = NULL, uint32_t timeout = 200000); void get_transfers(wallet2::transfer_container& incoming_transfers) const; void get_payments(const crypto::hash& payment_id, std::list<wallet2::payment_details>& payments, uint64_t min_height = 0, const boost::optional<uint32_t>& subaddr_account = boost::none, const std::set<uint32_t>& subaddr_indices = {}) const; void get_payments(std::list<std::pair<crypto::hash,wallet2::payment_details>>& payments, uint64_t min_height, uint64_t max_height = (uint64_t)-1, const boost::optional<uint32_t>& subaddr_account = boost::none, const std::set<uint32_t>& subaddr_indices = {}) const; @@ -1292,7 +1296,7 @@ namespace tools bool tx_add_fake_output(std::vector<std::vector<tools::wallet2::get_outs_entry>> &outs, uint64_t global_index, const crypto::public_key& tx_public_key, const rct::key& mask, uint64_t real_index, bool unlocked) const; bool should_pick_a_second_output(bool use_rct, size_t n_transfers, const std::vector<size_t> &unused_transfers_indices, const std::vector<size_t> &unused_dust_indices) const; std::vector<size_t> get_only_rct(const std::vector<size_t> &unused_dust_indices, const std::vector<size_t> &unused_transfers_indices) const; - void scan_output(const cryptonote::transaction &tx, const crypto::public_key &tx_pub_key, size_t i, tx_scan_info_t &tx_scan_info, int &num_vouts_received, std::unordered_map<cryptonote::subaddress_index, uint64_t> &tx_money_got_in_outs, std::vector<size_t> &outs, bool pool); + void scan_output(const cryptonote::transaction &tx, bool miner_tx, const crypto::public_key &tx_pub_key, size_t i, tx_scan_info_t &tx_scan_info, int &num_vouts_received, std::unordered_map<cryptonote::subaddress_index, uint64_t> &tx_money_got_in_outs, std::vector<size_t> &outs, bool pool); void trim_hashchain(); crypto::key_image get_multisig_composite_key_image(size_t n) const; rct::multisig_kLRki get_multisig_composite_kLRki(size_t n, const std::unordered_set<crypto::public_key> &ignore_set, std::unordered_set<rct::key> &used_L, std::unordered_set<rct::key> &new_used_L) const; diff --git a/src/wallet/wallet_rpc_server.cpp b/src/wallet/wallet_rpc_server.cpp index c87c2fca6..385e23818 100644 --- a/src/wallet/wallet_rpc_server.cpp +++ b/src/wallet/wallet_rpc_server.cpp @@ -63,6 +63,10 @@ namespace const command_line::arg_descriptor<bool> arg_restricted = {"restricted-rpc", "Restricts to view-only commands", false}; const command_line::arg_descriptor<std::string> arg_wallet_dir = {"wallet-dir", "Directory for newly created wallets"}; const command_line::arg_descriptor<bool> arg_prompt_for_password = {"prompt-for-password", "Prompts for password when not provided", false}; + const command_line::arg_descriptor<std::string> arg_rpc_ssl = {"rpc-ssl", tools::wallet2::tr("Enable SSL on wallet RPC connections: enabled|disabled|autodetect"), "autodetect"}; + const command_line::arg_descriptor<std::string> arg_rpc_ssl_private_key = {"rpc-ssl-private-key", tools::wallet2::tr("Path to a PEM format private key"), ""}; + const command_line::arg_descriptor<std::string> arg_rpc_ssl_certificate = {"rpc-ssl-certificate", tools::wallet2::tr("Path to a PEM format certificate"), ""}; + const command_line::arg_descriptor<std::vector<std::string>> arg_rpc_ssl_allowed_certificates = {"rpc-ssl-allowed-certificates", tools::wallet2::tr("List of paths to PEM format certificates of allowed RPC servers (all allowed if empty)")}; constexpr const char default_rpc_username[] = "monero"; @@ -233,10 +237,32 @@ namespace tools assert(bool(http_login)); } // end auth enabled + auto rpc_ssl_private_key = command_line::get_arg(vm, arg_rpc_ssl_private_key); + auto rpc_ssl_certificate = command_line::get_arg(vm, arg_rpc_ssl_certificate); + auto rpc_ssl_allowed_certificates = command_line::get_arg(vm, arg_rpc_ssl_allowed_certificates); + auto rpc_ssl = command_line::get_arg(vm, arg_rpc_ssl); + epee::net_utils::ssl_support_t rpc_ssl_support; + if (!epee::net_utils::ssl_support_from_string(rpc_ssl_support, rpc_ssl)) + { + MERROR("Invalid argument for " << std::string(arg_rpc_ssl.name)); + return false; + } + std::list<std::string> allowed_certificates; + for (const std::string &path: rpc_ssl_allowed_certificates) + { + allowed_certificates.push_back({}); + if (!epee::file_io_utils::load_file_to_string(path, allowed_certificates.back())) + { + MERROR("Failed to load certificate: " << path); + allowed_certificates.back() = std::string(); + } + } + m_net_server.set_threads_prefix("RPC"); auto rng = [](size_t len, uint8_t *ptr) { return crypto::rand(len, ptr); }; return epee::http_server_impl_base<wallet_rpc_server, connection_context>::init( - rng, std::move(bind_port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login) + rng, std::move(bind_port), std::move(rpc_config->bind_ip), std::move(rpc_config->access_control_origins), std::move(http_login), + rpc_ssl_support, std::make_pair(rpc_ssl_private_key, rpc_ssl_certificate), allowed_certificates ); } //------------------------------------------------------------------------------------------------------------------------------ @@ -691,13 +717,9 @@ namespace tools if (wallet2::parse_long_payment_id(payment_id_str, long_payment_id)) { cryptonote::set_payment_id_to_tx_extra_nonce(extra_nonce, long_payment_id); } - /* or short payment ID */ - else if (wallet2::parse_short_payment_id(payment_id_str, short_payment_id)) { - cryptonote::set_encrypted_payment_id_to_tx_extra_nonce(extra_nonce, short_payment_id); - } else { er.code = WALLET_RPC_ERROR_CODE_WRONG_PAYMENT_ID; - er.message = "Payment id has invalid format: \"" + payment_id_str + "\", expected 16 or 64 character string"; + er.message = "Payment id has invalid format: \"" + payment_id_str + "\", expected 64 character string"; return false; } @@ -3937,6 +3959,10 @@ int main(int argc, char** argv) { command_line::add_arg(desc_params, arg_from_json); command_line::add_arg(desc_params, arg_wallet_dir); command_line::add_arg(desc_params, arg_prompt_for_password); + command_line::add_arg(desc_params, arg_rpc_ssl); + command_line::add_arg(desc_params, arg_rpc_ssl_private_key); + command_line::add_arg(desc_params, arg_rpc_ssl_certificate); + command_line::add_arg(desc_params, arg_rpc_ssl_allowed_certificates); daemonizer::init_options(hidden_options, desc_params); desc_params.add(hidden_options); diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 20afd4203..89d77cf32 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -85,6 +85,7 @@ add_subdirectory(performance_tests) add_subdirectory(core_proxy) add_subdirectory(unit_tests) add_subdirectory(difficulty) +add_subdirectory(block_weight) add_subdirectory(hash) add_subdirectory(net_load_tests) if (BUILD_GUI_DEPS) @@ -115,6 +116,7 @@ add_test( set(enabled_tests core_tests difficulty + block_weight hash performance_tests core_proxy diff --git a/tests/block_weight/CMakeLists.txt b/tests/block_weight/CMakeLists.txt new file mode 100644 index 000000000..b0d716ea0 --- /dev/null +++ b/tests/block_weight/CMakeLists.txt @@ -0,0 +1,45 @@ +# Copyright (c) 2014-2018, The Monero Project +# +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without modification, are +# permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this list of +# conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, this list +# of conditions and the following disclaimer in the documentation and/or other +# materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its contributors may be +# used to endorse or promote products derived from this software without specific +# prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set(block_weight_sources + block_weight.cpp) + +set(block_weight_headers) + +add_executable(block_weight + ${block_weight_sources} + ${block_weight_headers}) +target_link_libraries(block_weight + PRIVATE + cryptonote_core + blockchain_db + ${EXTRA_LIBRARIES}) + +add_test( + NAME block_weight + COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/compare.py ${CMAKE_CURRENT_SOURCE_DIR}/block_weight.py ${CMAKE_CURRENT_BINARY_DIR}/block_weight) diff --git a/tests/block_weight/block_weight.cpp b/tests/block_weight/block_weight.cpp new file mode 100644 index 000000000..a67f40f0f --- /dev/null +++ b/tests/block_weight/block_weight.cpp @@ -0,0 +1,183 @@ +// Copyright (c) 2019, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#define IN_UNIT_TESTS + +#include <stdio.h> +#include <math.h> +#include "cryptonote_core/blockchain.h" +#include "cryptonote_core/tx_pool.h" +#include "cryptonote_core/cryptonote_core.h" +#include "blockchain_db/testdb.h" + +#define LONG_TERM_BLOCK_WEIGHT_WINDOW 5000 + +enum test_t +{ + test_max = 0, + test_lcg = 1, + test_min = 2, +}; + +namespace +{ + +class TestDB: public cryptonote::BaseTestDB +{ +private: + struct block_t + { + size_t weight; + uint64_t long_term_weight; + }; + +public: + TestDB() { m_open = true; } + + virtual void add_block( const cryptonote::block& blk + , size_t block_weight + , uint64_t long_term_block_weight + , const cryptonote::difficulty_type& cumulative_difficulty + , const uint64_t& coins_generated + , uint64_t num_rct_outs + , const crypto::hash& blk_hash + ) override { + blocks.push_back({block_weight, long_term_block_weight}); + } + virtual uint64_t height() const override { return blocks.size(); } + virtual size_t get_block_weight(const uint64_t &h) const override { return blocks[h].weight; } + virtual uint64_t get_block_long_term_weight(const uint64_t &h) const override { return blocks[h].long_term_weight; } + virtual crypto::hash top_block_hash() const override { + uint64_t h = height(); + crypto::hash top = crypto::null_hash; + if (h) + *(uint64_t*)&top = h - 1; + return top; + } + virtual void pop_block(cryptonote::block &blk, std::vector<cryptonote::transaction> &txs) override { blocks.pop_back(); } + virtual void set_hard_fork_version(uint64_t height, uint8_t version) override { if (height >= hf.size()) hf.resize(height + 1); hf[height] = version; } + virtual uint8_t get_hard_fork_version(uint64_t height) const override { if (height >= hf.size()) return 255; return hf[height]; } + +private: + std::vector<block_t> blocks; + std::vector<uint8_t> hf; +}; + +} + +#define PREFIX_WINDOW(hf_version,window) \ + std::unique_ptr<cryptonote::Blockchain> bc; \ + cryptonote::tx_memory_pool txpool(*bc); \ + bc.reset(new cryptonote::Blockchain(txpool)); \ + struct get_test_options { \ + const std::pair<uint8_t, uint64_t> hard_forks[3]; \ + const cryptonote::test_options test_options = { \ + hard_forks, \ + window, \ + }; \ + get_test_options(): hard_forks{std::make_pair(1, (uint64_t)0), std::make_pair((uint8_t)hf_version, (uint64_t)LONG_TERM_BLOCK_WEIGHT_WINDOW), std::make_pair((uint8_t)0, (uint64_t)0)} {} \ + } opts; \ + cryptonote::Blockchain *blockchain = bc.get(); \ + bool r = blockchain->init(new TestDB(), cryptonote::FAKECHAIN, true, &opts.test_options, 0, NULL); \ + if (!r) \ + { \ + fprintf(stderr, "Failed to init blockchain\n"); \ + exit(1); \ + } + +#define PREFIX(hf_version) PREFIX_WINDOW(hf_version, LONG_TERM_BLOCK_WEIGHT_WINDOW) + +static uint32_t lcg_seed = 0; + +static uint32_t lcg() +{ + lcg_seed = (lcg_seed * 0x100000001b3 + 0xcbf29ce484222325) & 0xffffffff; + return lcg_seed; +} + +static void test(test_t t, uint64_t blocks) +{ + PREFIX(10); + + for (uint64_t h = 0; h < LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + cryptonote::block b; + b.major_version = 1; + b.minor_version = 1; + bc->get_db().add_block(b, 300000, 300000, bc->get_db().height(), bc->get_db().height(), {}); + if (!bc->update_next_cumulative_weight_limit()) + { + fprintf(stderr, "Failed to update cumulative weight limit 1\n"); + exit(1); + } + } + + for (uint64_t h = 0; h < blocks; ++h) + { + uint64_t w; + uint64_t effective_block_weight_median = bc->get_current_cumulative_block_weight_median(); + switch (t) + { + case test_lcg: + { + uint32_t r = lcg(); + int64_t wi = 90 + r % 500000 + 250000 + sin(h / 200.) * 350000; + w = wi < 90 ? 90 : wi; + break; + } + case test_max: + w = bc->get_current_cumulative_block_weight_limit(); + break; + case test_min: + w = 90; + break; + default: + exit(1); + } + uint64_t ltw = bc->get_next_long_term_block_weight(w); + cryptonote::block b; + b.major_version = 10; + b.minor_version = 10; + bc->get_db().add_block(std::move(b), w, ltw, bc->get_db().height(), bc->get_db().height(), {}); + + if (!bc->update_next_cumulative_weight_limit()) + { + fprintf(stderr, "Failed to update cumulative weight limit\n"); + exit(1); + } + std::cout << "H " << h << ", BW " << w << ", EMBW " << effective_block_weight_median << ", LTBW " << ltw << std::endl; + } +} + +int main() +{ + test(test_max, 2 * LONG_TERM_BLOCK_WEIGHT_WINDOW); + test(test_lcg, 9 * LONG_TERM_BLOCK_WEIGHT_WINDOW); + test(test_min, 1 * LONG_TERM_BLOCK_WEIGHT_WINDOW); + return 0; +} diff --git a/tests/block_weight/block_weight.py b/tests/block_weight/block_weight.py new file mode 100755 index 000000000..06aaabb02 --- /dev/null +++ b/tests/block_weight/block_weight.py @@ -0,0 +1,74 @@ +#!/usr/bin/python +# Simulate a maximal block attack on the Monero network +# This uses the scheme proposed by ArticMine +# Written by Sarang Nother +# Copyright (c) 2019 The Monero Project +import sys +import math + +MEDIAN_WINDOW_SMALL = 100 # number of recent blocks for median computation +MEDIAN_WINDOW_BIG = 5000 +MULTIPLIER_SMALL = 1.4 # multipliers for determining weights +MULTIPLIER_BIG = 50.0 +MEDIAN_THRESHOLD = 300*1000 # initial value for median (scaled kB -> B) +lcg_seed = 0 +embw = MEDIAN_THRESHOLD +ltembw = MEDIAN_THRESHOLD + +weights = [MEDIAN_THRESHOLD]*MEDIAN_WINDOW_SMALL # weights of recent blocks (B), with index -1 most recent +lt_weights = [MEDIAN_THRESHOLD]*MEDIAN_WINDOW_BIG # long-term weights + +# Compute the median of a list +def get_median(vec): + #temp = vec + temp = sorted(vec) + if len(temp) % 2 == 1: + return temp[len(temp)/2] + else: + return int((temp[len(temp)/2]+temp[len(temp)/2-1])/2) + +def LCG(): + global lcg_seed + lcg_seed = (lcg_seed * 0x100000001b3 + 0xcbf29ce484222325) & 0xffffffff + return lcg_seed + +def run(t, blocks): + global embw + global ltembw + + weights = [MEDIAN_THRESHOLD]*MEDIAN_WINDOW_SMALL # weights of recent blocks (B), with index -1 most recent + lt_weights = [MEDIAN_THRESHOLD]*MEDIAN_WINDOW_BIG # long-term weights + + for block in range(blocks): + # determine the long-term effective weight + ltmedian = get_median(lt_weights[-MEDIAN_WINDOW_BIG:]) + ltembw = max(MEDIAN_THRESHOLD,ltmedian) + + # determine the effective weight + stmedian = get_median(weights[-MEDIAN_WINDOW_SMALL:]) + embw = min(max(MEDIAN_THRESHOLD,stmedian),int(MULTIPLIER_BIG*ltembw)) + + # drop the lowest values + weights = weights[1:] + lt_weights = lt_weights[1:] + + # add a block of max weight + if t == 0: + max_weight = 2 * embw + elif t == 1: + r = LCG() + max_weight = int(90 + r % 500000 + 250000 + math.sin(block / 200.) * 350000) + if max_weight < 90: max_weight = 90 + elif t == 2: + max_weight = 90 + else: + sys.exit(1) + weights.append(max_weight) + lt_weights.append(min(max_weight,int(ltembw + int(ltembw * 2 / 5)))) + + #print "H %u, r %u, BW %u, EMBW %u, LTBW %u, LTEMBW %u, ltmedian %u" % (block, r, max_weight, embw, lt_weights[-1], ltembw, ltmedian) + print "H %u, BW %u, EMBW %u, LTBW %u" % (block, max_weight, embw, lt_weights[-1]) + +run(0, 2 * MEDIAN_WINDOW_BIG) +run(1, 9 * MEDIAN_WINDOW_BIG) +run(2, 1 * MEDIAN_WINDOW_BIG) diff --git a/tests/block_weight/compare.py b/tests/block_weight/compare.py new file mode 100755 index 000000000..c6be05206 --- /dev/null +++ b/tests/block_weight/compare.py @@ -0,0 +1,13 @@ +#!/usr/bin/python + +import sys +import subprocess + +print 'running: ', sys.argv[1] +S0 = subprocess.check_output(sys.argv[1], stderr=subprocess.STDOUT) +print 'running: ', sys.argv[2] +S1 = subprocess.check_output(sys.argv[2], stderr=subprocess.STDOUT) +print 'comparing' +if S0 != S1: + sys.exit(1) +sys.exit(0) diff --git a/tests/core_tests/v2_tests.h b/tests/core_tests/v2_tests.h index ecb23818c..cd2bcb839 100644 --- a/tests/core_tests/v2_tests.h +++ b/tests/core_tests/v2_tests.h @@ -81,7 +81,7 @@ template<> struct get_test_options<gen_v2_tx_validation_base> { const std::pair<uint8_t, uint64_t> hard_forks[3] = {std::make_pair(1, 0), std::make_pair(2, 1), std::make_pair(0, 0)}; const cryptonote::test_options test_options = { - hard_forks + hard_forks, 0 }; }; diff --git a/tests/crypto/CMakeLists.txt b/tests/crypto/CMakeLists.txt index df96c57cc..8789cb552 100644 --- a/tests/crypto/CMakeLists.txt +++ b/tests/crypto/CMakeLists.txt @@ -52,3 +52,17 @@ set_property(TARGET cncrypto-tests add_test( NAME cncrypto COMMAND cncrypto-tests "${CMAKE_CURRENT_SOURCE_DIR}/tests.txt") + +add_executable(cnv4-jit-tests cnv4-jit.c) +target_link_libraries(cnv4-jit-tests + PRIVATE + crypto + common + ${EXTRA_LIBRARIES}) +set_property(TARGET cnv4-jit-tests + PROPERTY + FOLDER "tests") + +add_test( + NAME cnv4-jit + COMMAND cnv4-jit-tests 1788000 1789000) diff --git a/tests/crypto/cnv4-jit.c b/tests/crypto/cnv4-jit.c new file mode 100644 index 000000000..0f11e4393 --- /dev/null +++ b/tests/crypto/cnv4-jit.c @@ -0,0 +1,119 @@ +// Copyright (c) 2019, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include <stdint.h> +#include <stddef.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include "crypto/hash-ops.h" + +extern volatile int use_v4_jit_flag; + +static int test(const uint8_t *data, size_t len, uint64_t height) +{ + char hash0[32], hash1[32]; + use_v4_jit_flag = 0; + cn_slow_hash(data, len, hash0, 4, 0, height); + use_v4_jit_flag = 1; + cn_slow_hash(data, len, hash1, 4, 0, height); + return memcmp(hash0, hash1, 32); +} + +int main(int argc, char **argv) +{ + uint8_t data[64]; + uint64_t start_height = 1788000; + uint64_t end_height = 1788001; + + if (argc != 1 && argc != 2 && argc != 3) + { + fprintf(stderr, "usage: %s [<start_height> [<end_height>]]\n", argv[0]); + return 1; + } + if (argc > 1) + { + errno = 0; + start_height = strtoull(argv[1], NULL, 10); + if ((start_height == 0 && errno) || start_height == ULLONG_MAX) + { + fprintf(stderr, "invalid start_height\n"); + return 1; + } + end_height = start_height; + if (argc > 2) + { + errno = 0; + end_height = strtoull(argv[2], NULL, 10); + if ((end_height == 0 && errno) || end_height == ULLONG_MAX) + { + fprintf(stderr, "invalid end_height\n"); + return 1; + } + } + } + + if (start_height == end_height) + { + uint64_t counter = 0; + while (1) + { + printf("\r%llu", (unsigned long long)counter); + fflush(stdout); + size_t offset = 0; + while (offset + 8 < sizeof(data)) + { + memcpy(data + offset, &counter, sizeof(counter)); + offset += 8; + } + if (test(data, sizeof(data), start_height)) + { + fprintf(stderr, "\nFailure at height %llu, counter %llu\n", (unsigned long long)start_height, (unsigned long long)counter); + return 0; + } + ++counter; + } + } + + memset(data, 0x42, sizeof(data)); + for (uint64_t h = start_height; h < end_height; ++h) + { + printf("\r%llu/%llu", (unsigned long long)(h-start_height), (unsigned long long)(end_height-start_height)); + fflush(stdout); + if (test(data, sizeof(data), h)) + { + fprintf(stderr, "\nFailure at height %llu\n", (unsigned long long)h); + return 0; + } + } + + printf("\r"); + + return 0; +} diff --git a/tests/hash/CMakeLists.txt b/tests/hash/CMakeLists.txt index 433cf94e9..105cf2c13 100644 --- a/tests/hash/CMakeLists.txt +++ b/tests/hash/CMakeLists.txt @@ -43,7 +43,7 @@ set_property(TARGET hash-tests PROPERTY FOLDER "tests") -foreach (hash IN ITEMS fast slow slow-1 slow-2 tree extra-blake extra-groestl extra-jh extra-skein) +foreach (hash IN ITEMS fast slow slow-1 slow-2 slow-4 tree extra-blake extra-groestl extra-jh extra-skein) add_test( NAME "hash-${hash}" COMMAND hash-tests "${hash}" "${CMAKE_CURRENT_SOURCE_DIR}/tests-${hash}.txt") diff --git a/tests/hash/main.cpp b/tests/hash/main.cpp index 7767d0d3b..acfd99e96 100644 --- a/tests/hash/main.cpp +++ b/tests/hash/main.cpp @@ -44,6 +44,13 @@ using namespace std; using namespace crypto; typedef crypto::hash chash; +struct V4_Data +{ + const void* data; + size_t length; + uint64_t height; +}; + PUSH_WARNINGS DISABLE_VS_WARNINGS(4297) extern "C" { @@ -54,13 +61,17 @@ extern "C" { tree_hash((const char (*)[crypto::HASH_SIZE]) data, length >> 5, hash); } static void cn_slow_hash_0(const void *data, size_t length, char *hash) { - return cn_slow_hash(data, length, hash, 0/*variant*/, 0/*prehashed*/); + return cn_slow_hash(data, length, hash, 0/*variant*/, 0/*prehashed*/, 0/*height*/); } static void cn_slow_hash_1(const void *data, size_t length, char *hash) { - return cn_slow_hash(data, length, hash, 1/*variant*/, 0/*prehashed*/); + return cn_slow_hash(data, length, hash, 1/*variant*/, 0/*prehashed*/, 0/*height*/); } static void cn_slow_hash_2(const void *data, size_t length, char *hash) { - return cn_slow_hash(data, length, hash, 2/*variant*/, 0/*prehashed*/); + return cn_slow_hash(data, length, hash, 2/*variant*/, 0/*prehashed*/, 0/*height*/); + } + static void cn_slow_hash_4(const void *data, size_t, char *hash) { + const V4_Data* p = reinterpret_cast<const V4_Data*>(data); + return cn_slow_hash(p->data, p->length, hash, 4/*variant*/, 0/*prehashed*/, p->height); } } POP_WARNINGS @@ -72,7 +83,7 @@ struct hash_func { } hashes[] = {{"fast", cn_fast_hash}, {"slow", cn_slow_hash_0}, {"tree", hash_tree}, {"extra-blake", hash_extra_blake}, {"extra-groestl", hash_extra_groestl}, {"extra-jh", hash_extra_jh}, {"extra-skein", hash_extra_skein}, - {"slow-1", cn_slow_hash_1}, {"slow-2", cn_slow_hash_2}}; + {"slow-1", cn_slow_hash_1}, {"slow-2", cn_slow_hash_2}, {"slow-4", cn_slow_hash_4}}; int test_variant2_int_sqrt(); int test_variant2_int_sqrt_ref(); @@ -140,7 +151,15 @@ int main(int argc, char *argv[]) { input.exceptions(ios_base::badbit | ios_base::failbit | ios_base::eofbit); input.clear(input.rdstate()); get(input, data); - f(data.data(), data.size(), (char *) &actual); + if (f == cn_slow_hash_4) { + V4_Data d; + d.data = data.data(); + d.length = data.size(); + get(input, d.height); + f(&d, 0, (char *) &actual); + } else { + f(data.data(), data.size(), (char *) &actual); + } if (expected != actual) { size_t i; cerr << "Hash mismatch on test " << test << endl << "Input: "; diff --git a/tests/hash/tests-slow-4.txt b/tests/hash/tests-slow-4.txt new file mode 100644 index 000000000..06ab52cff --- /dev/null +++ b/tests/hash/tests-slow-4.txt @@ -0,0 +1,10 @@ +f759588ad57e758467295443a9bd71490abff8e9dad1b95b6bf2f5d0d78387bc 5468697320697320612074657374205468697320697320612074657374205468697320697320612074657374 1806260 +5bb833deca2bdd7252a9ccd7b4ce0b6a4854515794b56c207262f7a5b9bdb566 4c6f72656d20697073756d20646f6c6f722073697420616d65742c20636f6e73656374657475722061646970697363696e67 1806261 +1ee6728da60fbd8d7d55b2b1ade487a3cf52a2c3ac6f520db12c27d8921f6cab 656c69742c2073656420646f20656975736d6f642074656d706f7220696e6369646964756e74207574206c61626f7265 1806262 +6969fe2ddfb758438d48049f302fc2108a4fcc93e37669170e6db4b0b9b4c4cb 657420646f6c6f7265206d61676e6120616c697175612e20557420656e696d206164206d696e696d2076656e69616d2c 1806263 +7f3048b4e90d0cbe7a57c0394f37338a01fae3adfdc0e5126d863a895eb04e02 71756973206e6f737472756420657865726369746174696f6e20756c6c616d636f206c61626f726973206e697369 1806264 +1d290443a4b542af04a82f6b2494a6ee7f20f2754c58e0849032483a56e8e2ef 757420616c697175697020657820656120636f6d6d6f646f20636f6e7365717561742e20447569732061757465 1806265 +c43cc6567436a86afbd6aa9eaa7c276e9806830334b614b2bee23cc76634f6fd 697275726520646f6c6f7220696e20726570726568656e646572697420696e20766f6c7570746174652076656c6974 1806266 +87be2479c0c4e8edfdfaa5603e93f4265b3f8224c1c5946feb424819d18990a4 657373652063696c6c756d20646f6c6f726520657520667567696174206e756c6c612070617269617475722e 1806267 +dd9d6a6d8e47465cceac0877ef889b93e7eba979557e3935d7f86dce11b070f3 4578636570746575722073696e74206f6363616563617420637570696461746174206e6f6e2070726f6964656e742c 1806268 +75c6f2ae49a20521de97285b431e717125847fb8935ed84a61e7f8d36a2c3d8e 73756e7420696e2063756c706120717569206f666669636961206465736572756e74206d6f6c6c697420616e696d20696420657374206c61626f72756d2e 1806269 diff --git a/tests/performance_tests/cn_slow_hash.h b/tests/performance_tests/cn_slow_hash.h index b484afa41..79ebb8778 100644 --- a/tests/performance_tests/cn_slow_hash.h +++ b/tests/performance_tests/cn_slow_hash.h @@ -34,6 +34,7 @@ #include "crypto/crypto.h" #include "cryptonote_basic/cryptonote_basic.h" +template<unsigned int variant> class test_cn_slow_hash { public: @@ -42,18 +43,15 @@ public: #pragma pack(push, 1) struct data_t { - char data[13]; + char data[43]; }; #pragma pack(pop) - static_assert(13 == sizeof(data_t), "Invalid structure size"); + static_assert(43 == sizeof(data_t), "Invalid structure size"); bool init() { - if (!epee::string_tools::hex_to_pod("63617665617420656d70746f72", m_data)) - return false; - - if (!epee::string_tools::hex_to_pod("bbec2cacf69866a8e740380fe7b818fc78f8571221742d729d9d02d7f8989b87", m_expected_hash)) + if (!epee::string_tools::hex_to_pod("63617665617420656d70746f763617665617420656d70746f72263617665617420656d70746f7201020304", m_data)) return false; return true; @@ -62,11 +60,10 @@ public: bool test() { crypto::hash hash; - crypto::cn_slow_hash(&m_data, sizeof(m_data), hash); - return hash == m_expected_hash; + crypto::cn_slow_hash(&m_data, sizeof(m_data), hash, variant); + return true; } private: data_t m_data; - crypto::hash m_expected_hash; }; diff --git a/tests/performance_tests/main.cpp b/tests/performance_tests/main.cpp index bfe6de895..22a86cb59 100644 --- a/tests/performance_tests/main.cpp +++ b/tests/performance_tests/main.cpp @@ -77,10 +77,12 @@ int main(int argc, char** argv) const command_line::arg_descriptor<bool> arg_verbose = { "verbose", "Verbose output", false }; const command_line::arg_descriptor<bool> arg_stats = { "stats", "Including statistics (min/median)", false }; const command_line::arg_descriptor<unsigned> arg_loop_multiplier = { "loop-multiplier", "Run for that many times more loops", 1 }; + const command_line::arg_descriptor<std::string> arg_timings_database = { "timings-database", "Keep timings history in a file" }; command_line::add_arg(desc_options, arg_filter); command_line::add_arg(desc_options, arg_verbose); command_line::add_arg(desc_options, arg_stats); command_line::add_arg(desc_options, arg_loop_multiplier); + command_line::add_arg(desc_options, arg_timings_database); po::variables_map vm; bool r = command_line::handle_error_helper(desc_options, [&]() @@ -93,7 +95,10 @@ int main(int argc, char** argv) return 1; const std::string filter = tools::glob_to_regex(command_line::get_arg(vm, arg_filter)); + const std::string timings_database = command_line::get_arg(vm, arg_timings_database); Params p; + if (!timings_database.empty()) + p.td = TimingsDatabase(timings_database); p.verbose = command_line::get_arg(vm, arg_verbose); p.stats = command_line::get_arg(vm, arg_stats); p.loop_multiplier = command_line::get_arg(vm, arg_loop_multiplier); @@ -193,7 +198,10 @@ int main(int argc, char** argv) TEST_PERFORMANCE2(filter, p, test_wallet2_expand_subaddresses, 50, 200); - TEST_PERFORMANCE0(filter, p, test_cn_slow_hash); + TEST_PERFORMANCE1(filter, p, test_cn_slow_hash, 0); + TEST_PERFORMANCE1(filter, p, test_cn_slow_hash, 1); + TEST_PERFORMANCE1(filter, p, test_cn_slow_hash, 2); + TEST_PERFORMANCE1(filter, p, test_cn_slow_hash, 4); TEST_PERFORMANCE1(filter, p, test_cn_fast_hash, 32); TEST_PERFORMANCE1(filter, p, test_cn_fast_hash, 16384); diff --git a/tests/performance_tests/performance_tests.h b/tests/performance_tests/performance_tests.h index d37dda729..17d16b0f6 100644 --- a/tests/performance_tests/performance_tests.h +++ b/tests/performance_tests/performance_tests.h @@ -37,7 +37,9 @@ #include <boost/regex.hpp> #include "misc_language.h" +#include "stats.h" #include "common/perf_timer.h" +#include "common/timings.h" class performance_timer { @@ -67,6 +69,7 @@ private: struct Params { + TimingsDatabase td; bool verbose; bool stats; unsigned loop_multiplier; @@ -85,6 +88,8 @@ public: bool run() { + static_assert(0 < T::loop_count, "T::loop_count must be greater than 0"); + T test; if (!test.init()) return false; @@ -106,11 +111,13 @@ public: m_per_call_timers[i].pause(); } m_elapsed = timer.elapsed_ms(); + m_stats.reset(new Stats<tools::PerformanceTimer, uint64_t>(m_per_call_timers)); return true; } int elapsed_time() const { return m_elapsed; } + size_t get_size() const { return m_stats->get_size(); } int time_per_call(int scale = 1) const { @@ -118,59 +125,19 @@ public: return m_elapsed * scale / (T::loop_count * m_params.loop_multiplier); } - uint64_t per_call_min() const - { - uint64_t v = std::numeric_limits<uint64_t>::max(); - for (const auto &pt: m_per_call_timers) - v = std::min(v, pt.value()); - return v; - } - - uint64_t per_call_max() const - { - uint64_t v = std::numeric_limits<uint64_t>::min(); - for (const auto &pt: m_per_call_timers) - v = std::max(v, pt.value()); - return v; - } - - uint64_t per_call_mean() const - { - uint64_t v = 0; - for (const auto &pt: m_per_call_timers) - v += pt.value(); - return v / m_per_call_timers.size(); - } - - uint64_t per_call_median() const - { - std::vector<uint64_t> values; - values.reserve(m_per_call_timers.size()); - for (const auto &pt: m_per_call_timers) - values.push_back(pt.value()); - return epee::misc_utils::median(values); - } + uint64_t get_min() const { return m_stats->get_min(); } + uint64_t get_max() const { return m_stats->get_max(); } + double get_mean() const { return m_stats->get_mean(); } + uint64_t get_median() const { return m_stats->get_median(); } + double get_stddev() const { return m_stats->get_standard_deviation(); } + double get_non_parametric_skew() const { return m_stats->get_non_parametric_skew(); } + std::vector<uint64_t> get_quantiles(size_t n) const { return m_stats->get_quantiles(n); } - uint64_t per_call_stddev() const + bool is_same_distribution(size_t npoints, double mean, double stddev) const { - if (m_per_call_timers.size() <= 1) - return 0; - const uint64_t mean = per_call_mean(); - uint64_t acc = 0; - for (const auto &pt: m_per_call_timers) - { - int64_t dv = pt.value() - mean; - acc += dv * dv; - } - acc /= m_per_call_timers.size () - 1; - return sqrt(acc); + return m_stats->is_same_distribution_99(npoints, mean, stddev); } - uint64_t min_time_ns() const { return tools::ticks_to_ns(per_call_min()); } - uint64_t max_time_ns() const { return tools::ticks_to_ns(per_call_max()); } - uint64_t median_time_ns() const { return tools::ticks_to_ns(per_call_median()); } - uint64_t standard_deviation_time_ns() const { return tools::ticks_to_ns(per_call_stddev()); } - private: /** * Warm up processor core, enabling turbo boost, etc. @@ -191,10 +158,11 @@ private: int m_elapsed; Params m_params; std::vector<tools::PerformanceTimer> m_per_call_timers; + std::unique_ptr<Stats<tools::PerformanceTimer, uint64_t>> m_stats; }; template <typename T> -void run_test(const std::string &filter, const Params ¶ms, const char* test_name) +void run_test(const std::string &filter, Params ¶ms, const char* test_name) { boost::smatch match; if (!filter.empty() && !boost::regex_match(std::string(test_name), match, boost::regex(filter))) @@ -210,10 +178,10 @@ void run_test(const std::string &filter, const Params ¶ms, const char* test_ std::cout << " elapsed: " << runner.elapsed_time() << " ms\n"; if (params.stats) { - std::cout << " min: " << runner.min_time_ns() << " ns\n"; - std::cout << " max: " << runner.max_time_ns() << " ns\n"; - std::cout << " median: " << runner.median_time_ns() << " ns\n"; - std::cout << " std dev: " << runner.standard_deviation_time_ns() << " ns\n"; + std::cout << " min: " << runner.get_min() << " ns\n"; + std::cout << " max: " << runner.get_max() << " ns\n"; + std::cout << " median: " << runner.get_median() << " ns\n"; + std::cout << " std dev: " << runner.get_stddev() << " ns\n"; } } else @@ -221,24 +189,48 @@ void run_test(const std::string &filter, const Params ¶ms, const char* test_ std::cout << test_name << " (" << T::loop_count * params.loop_multiplier << " calls) - OK:"; } const char *unit = "ms"; - uint64_t scale = 1000000; - int time_per_call = runner.time_per_call(); - if (time_per_call < 30000) { + double scale = 1000000; + uint64_t time_per_call = runner.time_per_call(); + if (time_per_call < 100) { + scale = 1000; time_per_call = runner.time_per_call(1000); #ifdef _WIN32 unit = "\xb5s"; #else unit = "µs"; #endif - scale = 1000; } + const auto quantiles = runner.get_quantiles(10); + double min = runner.get_min(); + double max = runner.get_max(); + double med = runner.get_median(); + double mean = runner.get_mean(); + double stddev = runner.get_stddev(); + double npskew = runner.get_non_parametric_skew(); + + std::vector<TimingsDatabase::instance> prev_instances = params.td.get(test_name); + params.td.add(test_name, {time(NULL), runner.get_size(), min, max, mean, med, stddev, npskew, quantiles}); + std::cout << (params.verbose ? " time per call: " : " ") << time_per_call << " " << unit << "/call" << (params.verbose ? "\n" : ""); if (params.stats) { - uint64_t min_ns = runner.min_time_ns() / scale; - uint64_t med_ns = runner.median_time_ns() / scale; - uint64_t stddev_ns = runner.standard_deviation_time_ns() / scale; - std::cout << " (min " << min_ns << " " << unit << ", median " << med_ns << " " << unit << ", std dev " << stddev_ns << " " << unit << ")"; + uint64_t mins = min / scale; + uint64_t maxs = max / scale; + uint64_t meds = med / scale; + uint64_t p95s = quantiles[9] / scale; + uint64_t stddevs = stddev / scale; + std::string cmp; + if (!prev_instances.empty()) + { + const TimingsDatabase::instance &prev_instance = prev_instances.back(); + if (!runner.is_same_distribution(prev_instance.npoints, prev_instance.mean, prev_instance.stddev)) + { + double pc = fabs(100. * (prev_instance.mean - runner.get_mean()) / prev_instance.mean); + cmp = ", " + std::to_string(pc) + "% " + (mean > prev_instance.mean ? "slower" : "faster"); + } +cmp += " -- " + std::to_string(prev_instance.mean); + } + std::cout << " (min " << mins << " " << unit << ", 90th " << p95s << " " << unit << ", median " << meds << " " << unit << ", std dev " << stddevs << " " << unit << ")" << cmp; } std::cout << std::endl; } diff --git a/tests/unit_tests/CMakeLists.txt b/tests/unit_tests/CMakeLists.txt index 5f7fa84a5..aea82ede2 100644 --- a/tests/unit_tests/CMakeLists.txt +++ b/tests/unit_tests/CMakeLists.txt @@ -55,6 +55,7 @@ set(unit_tests_sources http.cpp keccak.cpp logging.cpp + long_term_block_weight.cpp main.cpp memwipe.cpp mlocker.cpp @@ -62,6 +63,7 @@ set(unit_tests_sources mul_div.cpp multiexp.cpp multisig.cpp + net.cpp notify.cpp output_distribution.cpp parse_amount.cpp @@ -100,6 +102,7 @@ target_link_libraries(unit_tests cryptonote_core blockchain_db rpc + net serialization wallet p2p diff --git a/tests/unit_tests/blockchain_db.cpp b/tests/unit_tests/blockchain_db.cpp index 7e7ce9bf7..9cf8c5fbe 100644 --- a/tests/unit_tests/blockchain_db.cpp +++ b/tests/unit_tests/blockchain_db.cpp @@ -277,10 +277,10 @@ TYPED_TEST(BlockchainDBTest, AddBlock) // TODO: need at least one more block to make this reasonable, as the // BlockchainDB implementation should not check for parent if // no blocks have been added yet (because genesis has no parent). - //ASSERT_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1]), BLOCK_PARENT_DNE); + //ASSERT_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1]), BLOCK_PARENT_DNE); - ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0])); - ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1])); + ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0])); + ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1])); block b; ASSERT_TRUE(this->m_db->block_exists(get_block_hash(this->m_blocks[0]))); @@ -293,7 +293,7 @@ TYPED_TEST(BlockchainDBTest, AddBlock) ASSERT_TRUE(compare_blocks(this->m_blocks[0], b)); // assert that we can't add the same block twice - ASSERT_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0]), TX_EXISTS); + ASSERT_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0]), TX_EXISTS); for (auto& h : this->m_blocks[0].tx_hashes) { @@ -317,14 +317,14 @@ TYPED_TEST(BlockchainDBTest, RetrieveBlockData) this->get_filenames(); this->init_hard_fork(); - ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0])); + ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[0], t_sizes[0], t_sizes[0], t_diffs[0], t_coins[0], this->m_txs[0])); ASSERT_EQ(t_sizes[0], this->m_db->get_block_weight(0)); ASSERT_EQ(t_diffs[0], this->m_db->get_block_cumulative_difficulty(0)); ASSERT_EQ(t_diffs[0], this->m_db->get_block_difficulty(0)); ASSERT_EQ(t_coins[0], this->m_db->get_block_already_generated_coins(0)); - ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1])); + ASSERT_NO_THROW(this->m_db->add_block(this->m_blocks[1], t_sizes[1], t_sizes[1], t_diffs[1], t_coins[1], this->m_txs[1])); ASSERT_EQ(t_diffs[1] - t_diffs[0], this->m_db->get_block_difficulty(1)); ASSERT_HASH_EQ(get_block_hash(this->m_blocks[0]), this->m_db->get_block_hash_from_height(0)); diff --git a/tests/unit_tests/epee_utils.cpp b/tests/unit_tests/epee_utils.cpp index 3d5882d7d..18fb262c2 100644 --- a/tests/unit_tests/epee_utils.cpp +++ b/tests/unit_tests/epee_utils.cpp @@ -545,6 +545,8 @@ TEST(StringTools, GetIpInt32) TEST(NetUtils, IPv4NetworkAddress) { + static_assert(epee::net_utils::ipv4_network_address::get_type_id() == epee::net_utils::address_type::ipv4, "bad ipv4 type id"); + const auto ip1 = boost::endian::native_to_big(0x330012FFu); const auto ip_loopback = boost::endian::native_to_big(0x7F000001u); const auto ip_local = boost::endian::native_to_big(0x0A000000u); @@ -555,7 +557,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip1, address1.ip()); EXPECT_EQ(65535, address1.port()); EXPECT_TRUE(epee::net_utils::ipv4_network_address{std::move(address1)} == address1); @@ -568,7 +570,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip_loopback, loopback.ip()); EXPECT_EQ(0, loopback.port()); @@ -624,7 +626,9 @@ TEST(NetUtils, NetworkAddress) constexpr static bool is_local() noexcept { return false; } static std::string str() { return {}; } static std::string host_str() { return {}; } - constexpr static uint8_t get_type_id() noexcept { return uint8_t(-1); } + constexpr static epee::net_utils::address_type get_type_id() noexcept { return epee::net_utils::address_type(-1); } + constexpr static epee::net_utils::zone get_zone() noexcept { return epee::net_utils::zone::invalid; } + constexpr static bool is_blockable() noexcept { return false; } }; const epee::net_utils::network_address empty; @@ -634,7 +638,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("<none>", empty.host_str().c_str()); EXPECT_FALSE(empty.is_loopback()); EXPECT_FALSE(empty.is_local()); - EXPECT_EQ(0, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::invalid, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::invalid, empty.get_zone()); + EXPECT_FALSE(empty.is_blockable()); EXPECT_THROW(empty.as<custom_address>(), std::bad_cast); epee::net_utils::network_address address1{ @@ -650,7 +656,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_TRUE(address1.is_blockable()); EXPECT_NO_THROW(address1.as<epee::net_utils::ipv4_network_address>()); EXPECT_THROW(address1.as<custom_address>(), std::bad_cast); @@ -667,7 +675,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); const epee::net_utils::network_address local{ epee::net_utils::ipv4_network_address{ip_local, 8080} diff --git a/tests/unit_tests/hardfork.cpp b/tests/unit_tests/hardfork.cpp index ec8d1d202..12dfde1bc 100644 --- a/tests/unit_tests/hardfork.cpp +++ b/tests/unit_tests/hardfork.cpp @@ -34,7 +34,7 @@ #include "blockchain_db/blockchain_db.h" #include "cryptonote_basic/cryptonote_format_utils.h" #include "cryptonote_basic/hardfork.h" -#include "testdb.h" +#include "blockchain_db/testdb.h" using namespace cryptonote; @@ -44,11 +44,12 @@ using namespace cryptonote; namespace { -class TestDB: public BaseTestDB { +class TestDB: public cryptonote::BaseTestDB { public: virtual uint64_t height() const { return blocks.size(); } virtual void add_block( const block& blk , size_t block_weight + , uint64_t long_term_block_weight , const difficulty_type& cumulative_difficulty , const uint64_t& coins_generated , uint64_t num_rct_outs @@ -106,20 +107,20 @@ TEST(major, Only) ASSERT_FALSE(hf.add(mkblock(0, 2), 0)); ASSERT_FALSE(hf.add(mkblock(2, 2), 0)); ASSERT_TRUE(hf.add(mkblock(1, 2), 0)); - db.add_block(mkblock(1, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(1, 1), 0, 0, 0, 0, 0, crypto::hash()); // block height 1, only version 1 is accepted ASSERT_FALSE(hf.add(mkblock(0, 2), 1)); ASSERT_FALSE(hf.add(mkblock(2, 2), 1)); ASSERT_TRUE(hf.add(mkblock(1, 2), 1)); - db.add_block(mkblock(1, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(1, 1), 0, 0, 0, 0, 0, crypto::hash()); // block height 2, only version 2 is accepted ASSERT_FALSE(hf.add(mkblock(0, 2), 2)); ASSERT_FALSE(hf.add(mkblock(1, 2), 2)); ASSERT_FALSE(hf.add(mkblock(3, 2), 2)); ASSERT_TRUE(hf.add(mkblock(2, 2), 2)); - db.add_block(mkblock(2, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(2, 1), 0, 0, 0, 0, 0, crypto::hash()); } TEST(empty_hardforks, Success) @@ -133,7 +134,7 @@ TEST(empty_hardforks, Success) ASSERT_TRUE(hf.get_state(time(NULL) + 3600*24*400) == HardFork::Ready); for (uint64_t h = 0; h <= 10; ++h) { - db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } ASSERT_EQ(hf.get(0), 1); @@ -167,14 +168,14 @@ TEST(check_for_height, Success) for (uint64_t h = 0; h <= 4; ++h) { ASSERT_TRUE(hf.check_for_height(mkblock(1, 1), h)); ASSERT_FALSE(hf.check_for_height(mkblock(2, 2), h)); // block version is too high - db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } for (uint64_t h = 5; h <= 10; ++h) { ASSERT_FALSE(hf.check_for_height(mkblock(1, 1), h)); // block version is too low ASSERT_TRUE(hf.check_for_height(mkblock(2, 2), h)); - db.add_block(mkblock(hf, h, 2), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 2), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } } @@ -191,19 +192,19 @@ TEST(get, next_version) for (uint64_t h = 0; h <= 4; ++h) { ASSERT_EQ(2, hf.get_next_version()); - db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 1), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } for (uint64_t h = 5; h <= 9; ++h) { ASSERT_EQ(4, hf.get_next_version()); - db.add_block(mkblock(hf, h, 2), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 2), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } for (uint64_t h = 10; h <= 15; ++h) { ASSERT_EQ(4, hf.get_next_version()); - db.add_block(mkblock(hf, h, 4), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 4), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } } @@ -244,7 +245,7 @@ TEST(steps_asap, Success) hf.init(); for (uint64_t h = 0; h < 10; ++h) { - db.add_block(mkblock(hf, h, 9), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, 9), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } @@ -271,7 +272,7 @@ TEST(steps_1, Success) hf.init(); for (uint64_t h = 0 ; h < 10; ++h) { - db.add_block(mkblock(hf, h, h+1), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, h+1), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } @@ -296,7 +297,7 @@ TEST(reorganize, Same) // index 0 1 2 3 4 5 6 7 8 9 static const uint8_t block_versions[] = { 1, 1, 4, 4, 7, 7, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9 }; for (uint64_t h = 0; h < 20; ++h) { - db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } @@ -327,7 +328,7 @@ TEST(reorganize, Changed) static const uint8_t block_versions[] = { 1, 1, 4, 4, 7, 7, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9 }; static const uint8_t expected_versions[] = { 1, 1, 1, 1, 1, 1, 4, 4, 7, 7, 9, 9, 9, 9, 9, 9 }; for (uint64_t h = 0; h < 16; ++h) { - db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE (hf.add(db.get_block_from_height(h), h)); } @@ -347,7 +348,7 @@ TEST(reorganize, Changed) ASSERT_EQ(db.height(), 3); hf.reorganize_from_block_height(2); for (uint64_t h = 3; h < 16; ++h) { - db.add_block(mkblock(hf, h, block_versions_new[h]), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, block_versions_new[h]), 0, 0, 0, 0, 0, crypto::hash()); bool ret = hf.add(db.get_block_from_height(h), h); ASSERT_EQ (ret, h < 15); } @@ -371,7 +372,7 @@ TEST(voting, threshold) for (uint64_t h = 0; h <= 8; ++h) { uint8_t v = 1 + !!(h % 8); - db.add_block(mkblock(hf, h, v), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, v), 0, 0, 0, 0, 0, crypto::hash()); bool ret = hf.add(db.get_block_from_height(h), h); if (h >= 8 && threshold == 87) { // for threshold 87, we reach the treshold at height 7, so from height 8, hard fork to version 2, but 8 tries to add 1 @@ -405,7 +406,7 @@ TEST(voting, different_thresholds) static const uint8_t expected_versions[] = { 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4 }; for (uint64_t h = 0; h < sizeof(block_versions) / sizeof(block_versions[0]); ++h) { - db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, 0, crypto::hash()); bool ret = hf.add(db.get_block_from_height(h), h); ASSERT_EQ(ret, true); } @@ -459,7 +460,7 @@ TEST(voting, info) ASSERT_EQ(expected_thresholds[h], threshold); ASSERT_EQ(4, voting); - db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, crypto::hash()); + db.add_block(mkblock(hf, h, block_versions[h]), 0, 0, 0, 0, 0, crypto::hash()); ASSERT_TRUE(hf.add(db.get_block_from_height(h), h)); } } @@ -522,7 +523,7 @@ TEST(reorganize, changed) #define ADD(v, h, a) \ do { \ cryptonote::block b = mkblock(hf, h, v); \ - db.add_block(b, 0, 0, 0, 0, crypto::hash()); \ + db.add_block(b, 0, 0, 0, 0, 0, crypto::hash()); \ ASSERT_##a(hf.add(b, h)); \ } while(0) #define ADD_TRUE(v, h) ADD(v, h, TRUE) diff --git a/tests/unit_tests/long_term_block_weight.cpp b/tests/unit_tests/long_term_block_weight.cpp new file mode 100644 index 000000000..0ee6c5c2b --- /dev/null +++ b/tests/unit_tests/long_term_block_weight.cpp @@ -0,0 +1,384 @@ +// Copyright (c) 2019, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#define IN_UNIT_TESTS + +#include "gtest/gtest.h" +#include "cryptonote_core/blockchain.h" +#include "cryptonote_core/tx_pool.h" +#include "cryptonote_core/cryptonote_core.h" +#include "blockchain_db/testdb.h" + +#define TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW 5000 + +namespace +{ + +class TestDB: public cryptonote::BaseTestDB +{ +private: + struct block_t + { + size_t weight; + uint64_t long_term_weight; + }; + +public: + TestDB() { m_open = true; } + + virtual void add_block( const cryptonote::block& blk + , size_t block_weight + , uint64_t long_term_block_weight + , const cryptonote::difficulty_type& cumulative_difficulty + , const uint64_t& coins_generated + , uint64_t num_rct_outs + , const crypto::hash& blk_hash + ) override { + blocks.push_back({block_weight, long_term_block_weight}); + } + virtual uint64_t height() const override { return blocks.size(); } + virtual size_t get_block_weight(const uint64_t &h) const override { return blocks[h].weight; } + virtual uint64_t get_block_long_term_weight(const uint64_t &h) const override { return blocks[h].long_term_weight; } + virtual crypto::hash top_block_hash() const override { + uint64_t h = height(); + crypto::hash top = crypto::null_hash; + if (h) + *(uint64_t*)&top = h - 1; + return top; + } + virtual void pop_block(cryptonote::block &blk, std::vector<cryptonote::transaction> &txs) override { blocks.pop_back(); } + +private: + std::vector<block_t> blocks; +}; + +static uint32_t lcg_seed = 0; + +static uint32_t lcg() +{ + lcg_seed = (lcg_seed * 0x100000001b3 + 0xcbf29ce484222325) & 0xffffffff; + return lcg_seed; +} + +} + +#define PREFIX_WINDOW(hf_version,window) \ + std::unique_ptr<cryptonote::Blockchain> bc; \ + cryptonote::tx_memory_pool txpool(*bc); \ + bc.reset(new cryptonote::Blockchain(txpool)); \ + struct get_test_options { \ + const std::pair<uint8_t, uint64_t> hard_forks[3]; \ + const cryptonote::test_options test_options = { \ + hard_forks, \ + window, \ + }; \ + get_test_options(): hard_forks{std::make_pair(1, (uint64_t)0), std::make_pair((uint8_t)hf_version, (uint64_t)1), std::make_pair((uint8_t)0, (uint64_t)0)} {} \ + } opts; \ + cryptonote::Blockchain *blockchain = bc.get(); \ + bool r = blockchain->init(new TestDB(), cryptonote::FAKECHAIN, true, &opts.test_options, 0, NULL); \ + ASSERT_TRUE(r) + +#define PREFIX(hf_version) PREFIX_WINDOW(hf_version, TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW) + +TEST(long_term_block_weight, empty_short) +{ + PREFIX(9); + + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + + ASSERT_EQ(bc->get_current_cumulative_block_weight_median(), CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5); + ASSERT_EQ(bc->get_current_cumulative_block_weight_limit(), CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 * 2); +} + +TEST(long_term_block_weight, identical_before_fork) +{ + PREFIX(9); + + for (uint64_t h = 1; h < 10 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + size_t w = h < CRYPTONOTE_REWARD_BLOCKS_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + for (uint64_t h = 0; h < 10 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + ASSERT_EQ(bc->get_db().get_block_long_term_weight(h), bc->get_db().get_block_weight(h)); + } +} + +TEST(long_term_block_weight, identical_after_fork_before_long_term_window) +{ + PREFIX(10); + + for (uint64_t h = 1; h <= TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + size_t w = h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + for (uint64_t h = 0; h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + ASSERT_EQ(bc->get_db().get_block_long_term_weight(h), bc->get_db().get_block_weight(h)); + } +} + +TEST(long_term_block_weight, ceiling_at_30000000) +{ + PREFIX(10); + + for (uint64_t h = 0; h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW + TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW / 2 - 1; ++h) + { + size_t w = h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + ASSERT_EQ(bc->get_current_cumulative_block_weight_median(), 15000000); + ASSERT_EQ(bc->get_current_cumulative_block_weight_limit(), 30000000); +} + +TEST(long_term_block_weight, multi_pop) +{ + PREFIX(10); + + for (uint64_t h = 1; h <= TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW + 20; ++h) + { + size_t w = h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + + const uint64_t effective_median = bc->get_current_cumulative_block_weight_median(); + const uint64_t effective_limit = bc->get_current_cumulative_block_weight_limit(); + + for (uint64_t h = 0; h < 4; ++h) + { + size_t w = h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + + cryptonote::block b; + std::vector<cryptonote::transaction> txs; + bc->get_db().pop_block(b, txs); + bc->get_db().pop_block(b, txs); + bc->get_db().pop_block(b, txs); + bc->get_db().pop_block(b, txs); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); +} + +TEST(long_term_block_weight, multiple_updates) +{ + PREFIX(10); + + for (uint64_t h = 1; h <= 3 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + size_t w = h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + const uint64_t effective_median = bc->get_current_cumulative_block_weight_median(); + const uint64_t effective_limit = bc->get_current_cumulative_block_weight_limit(); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); + } +} + +TEST(long_term_block_weight, pop_invariant_max) +{ + PREFIX(10); + + for (uint64_t h = 1; h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW - 10; ++h) + { + size_t w = bc->get_db().height() < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + + for (int n = 0; n < 1000; ++n) + { + // pop some blocks, then add some more + int remove = 1 + (n * 17) % 8; + int add = (n * 23) % 12; + + // save long term block weights we're about to remove + uint64_t old_ltbw[16], h0 = bc->get_db().height() - remove - 1; + for (int i = -2; i < remove; ++i) + { + old_ltbw[i + 2] = bc->get_db().get_block_long_term_weight(h0 + i); + } + + for (int i = 0; i < remove; ++i) + { + cryptonote::block b; + std::vector<cryptonote::transaction> txs; + bc->get_db().pop_block(b, txs); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + for (int i = 0; i < add; ++i) + { + size_t w = bc->get_db().height() < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, bc->get_db().height(), bc->get_db().height(), {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + + // check the new values are the same as the old ones + for (int i = -2; i < std::min(add, remove); ++i) + { + ASSERT_EQ(bc->get_db().get_block_long_term_weight(h0 + i), old_ltbw[i + 2]); + } + } +} + +TEST(long_term_block_weight, pop_invariant_random) +{ + PREFIX(10); + + for (uint64_t h = 1; h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW - 10; ++h) + { + size_t w = bc->get_db().height() < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + } + + for (int n = 0; n < 1000; ++n) + { + // pop some blocks, then add some more + int remove = 1 + (n * 17) % 8; + int add = (n * 23) % 123; + + // save long term block weights we're about to remove + uint64_t old_ltbw[16], h0 = bc->get_db().height() - remove - 1; + for (int i = -2; i < remove; ++i) + { + old_ltbw[i + 2] = bc->get_db().get_block_long_term_weight(h0 + i); + } + + for (int i = 0; i < remove; ++i) + { + cryptonote::block b; + std::vector<cryptonote::transaction> txs; + bc->get_db().pop_block(b, txs); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + const uint64_t effective_median = bc->get_current_cumulative_block_weight_median(); + const uint64_t effective_limit = bc->get_current_cumulative_block_weight_limit(); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); + } + for (int i = 0; i < add; ++i) + { + lcg_seed = bc->get_db().height(); + uint32_t r = lcg(); + size_t w = bc->get_db().height() < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW ? CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5 : (r % bc->get_current_cumulative_block_weight_limit()); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, bc->get_db().height(), bc->get_db().height(), {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + const uint64_t effective_median = bc->get_current_cumulative_block_weight_median(); + const uint64_t effective_limit = bc->get_current_cumulative_block_weight_limit(); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit()); + ASSERT_EQ(effective_median, bc->get_current_cumulative_block_weight_median()); + ASSERT_EQ(effective_limit, bc->get_current_cumulative_block_weight_limit()); + } + + // check the new values are the same as the old ones + for (int i = -2; i < std::min(add, remove); ++i) + { + ASSERT_EQ(bc->get_db().get_block_long_term_weight(h0 + i), old_ltbw[i + 2]); + } + } +} + +TEST(long_term_block_weight, long_growth_spike_and_drop) +{ + PREFIX(10); + + uint64_t long_term_effective_median_block_weight; + + // constant init + for (uint64_t h = 0; h < TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW; ++h) + { + size_t w = CRYPTONOTE_BLOCK_GRANTED_FULL_REWARD_ZONE_V5; + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit(&long_term_effective_median_block_weight)); + } + ASSERT_EQ(long_term_effective_median_block_weight, 300000); + + // slow 10% yearly for a year (scaled down by 100000 / TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW) -> 8% change + for (uint64_t h = 0; h < 365 * 720 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW / 100000; ++h) + { + //size_t w = bc->get_current_cumulative_block_weight_median() * rate; + float t = h / float(365 * 720 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW / 100000); + size_t w = 300000 + t * 30000; + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit(&long_term_effective_median_block_weight)); + } + ASSERT_GT(long_term_effective_median_block_weight, 300000 * 1.07); + ASSERT_LT(long_term_effective_median_block_weight, 300000 * 1.09); + + // spike over three weeks - does not move much + for (uint64_t h = 0; h < 21 * 720 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW / 100000; ++h) + { + size_t w = bc->get_current_cumulative_block_weight_limit(); + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit(&long_term_effective_median_block_weight)); + } + ASSERT_GT(long_term_effective_median_block_weight, 300000 * 1.07); + ASSERT_LT(long_term_effective_median_block_weight, 300000 * 1.09); + + // drop - does not move much + for (uint64_t h = 0; h < 21 * 720 * TEST_LONG_TERM_BLOCK_WEIGHT_WINDOW / 100000; ++h) + { + size_t w = bc->get_current_cumulative_block_weight_median() * .25; + uint64_t ltw = bc->get_next_long_term_block_weight(w); + bc->get_db().add_block(cryptonote::block(), w, ltw, h, h, {}); + ASSERT_TRUE(bc->update_next_cumulative_weight_limit(&long_term_effective_median_block_weight)); + } + ASSERT_GT(long_term_effective_median_block_weight, 300000 * 1.07); + ASSERT_LT(long_term_effective_median_block_weight, 300000 * 1.09); +} diff --git a/tests/unit_tests/net.cpp b/tests/unit_tests/net.cpp new file mode 100644 index 000000000..a38ecfe81 --- /dev/null +++ b/tests/unit_tests/net.cpp @@ -0,0 +1,745 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include <atomic> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/asio/buffer.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/system/error_code.hpp> +#include <boost/thread/thread.hpp> +#include <cstring> +#include <functional> +#include <gtest/gtest.h> +#include <memory> + +#include "net/error.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "p2p/net_peerlist_boost_serialization.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" + +namespace +{ + static constexpr const char v2_onion[] = + "xmrto2bturnore26.onion"; + static constexpr const char v3_onion[] = + "vww6ybal4bd7szmgncyruucpgfkqahzddi37ktceo3ah7ngmcopnpyyd.onion"; +} + +TEST(tor_address, constants) +{ + static_assert(!net::tor_address::is_local(), "bad is_local() response"); + static_assert(!net::tor_address::is_loopback(), "bad is_loopback() response"); + static_assert(net::tor_address::get_type_id() == epee::net_utils::address_type::tor, "bad get_type_id() response"); + + EXPECT_FALSE(net::tor_address::is_local()); + EXPECT_FALSE(net::tor_address::is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); +} + +TEST(tor_address, invalid) +{ + EXPECT_TRUE(net::tor_address::make("").has_error()); + EXPECT_TRUE(net::tor_address::make(":").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion:").has_error()); + EXPECT_TRUE(net::tor_address::make(v2_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(v3_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v2_onion, sizeof(v2_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v3_onion, sizeof(v3_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":-").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":900a").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":65536").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":-1").has_error()); + + std::string onion{v3_onion}; + onion.at(10) = 1; + EXPECT_TRUE(net::tor_address::make(onion).has_error()); +} + +TEST(tor_address, unblockable_types) +{ + net::tor_address tor{}; + + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + tor = net::tor_address::unknown(); + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + EXPECT_EQ(net::tor_address{}, net::tor_address::unknown()); +} + +TEST(tor_address, valid) +{ + const auto address1 = net::tor_address::make(v3_onion); + + ASSERT_TRUE(address1.has_value()); + EXPECT_EQ(0u, address1->port()); + EXPECT_STREQ(v3_onion, address1->host_str()); + EXPECT_STREQ(v3_onion, address1->str().c_str()); + EXPECT_TRUE(address1->is_blockable()); + + net::tor_address address2{*address1}; + + EXPECT_EQ(0u, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_STREQ(v3_onion, address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_TRUE(address2.equal(*address1)); + EXPECT_TRUE(address1->equal(address2)); + EXPECT_TRUE(address2 == *address1); + EXPECT_TRUE(*address1 == address2); + EXPECT_FALSE(address2 != *address1); + EXPECT_FALSE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_FALSE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v2_onion} + ":6545")); + + EXPECT_EQ(6545, address2.port()); + EXPECT_STREQ(v2_onion, address2.host_str()); + EXPECT_EQ(std::string{v2_onion} + ":6545", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_FALSE(address2.is_same_host(*address1)); + EXPECT_FALSE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v3_onion} + ":", 65535)); + + EXPECT_EQ(65535, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_EQ(std::string{v3_onion} + ":65535", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); +} + +TEST(tor_address, generic_network_address) +{ + const epee::net_utils::network_address tor1{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address tor2{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address ip{epee::net_utils::ipv4_network_address{100, 200}}; + + EXPECT_EQ(tor1, tor2); + EXPECT_NE(ip, tor1); + EXPECT_LT(ip, tor1); + + EXPECT_STREQ(v3_onion, tor1.host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":8080", tor1.str()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor1.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor2.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::ipv4, ip.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor1.get_zone()); + EXPECT_EQ(epee::net_utils::zone::tor, tor2.get_zone()); + EXPECT_EQ(epee::net_utils::zone::public_, ip.get_zone()); + EXPECT_TRUE(tor1.is_blockable()); + EXPECT_TRUE(tor2.is_blockable()); + EXPECT_TRUE(ip.is_blockable()); +} + +namespace +{ + struct test_command + { + net::tor_address tor; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(tor); + END_KV_SERIALIZE_MAP() + }; +} + +TEST(tor_address, epee_serializev_v2) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v2_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v2_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serializev_v3) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v3_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v3_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serialize_unknown) +{ + std::string buffer{}; + { + test_command command{net::tor_address::unknown()}; + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(net::tor_address::unknown_str()), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, boost_serialize_v2) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v2_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_v3) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v3_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_unknown) +{ + std::string buffer{}; + { + const net::tor_address tor{}; + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); +} + +TEST(get_network_address, onion) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("onion", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address(".onion", 0); + EXPECT_EQ(net::error::invalid_tor_address, address); + + address = net::get_network_address(v3_onion, 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":1000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":2000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":65536", 1000); + EXPECT_EQ(net::error::invalid_port, address); +} + + +TEST(get_network_address, ipv4) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("0.0.0.", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.257", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.254", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("0.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("0.0.0.254:1000", address->str().c_str()); + + address = net::get_network_address("23.0.0.254:2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("23.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("23.0.0.254:2000", address->str().c_str()); +} + +namespace +{ + using stream_type = boost::asio::ip::tcp; + + struct io_thread + { + boost::asio::io_service io_service; + boost::asio::io_service::work work; + stream_type::socket server; + stream_type::acceptor acceptor; + boost::thread io; + std::atomic<bool> connected; + + io_thread() + : io_service(), + work(io_service), + server(io_service), + acceptor(io_service), + io([this] () { try { this->io_service.run(); } catch (const std::exception& e) { MERROR(e.what()); }}), + connected(false) + { + acceptor.open(boost::asio::ip::tcp::v4()); + acceptor.bind(stream_type::endpoint{boost::asio::ip::tcp::v4(), 0}); + acceptor.listen(); + acceptor.async_accept(server, [this] (boost::system::error_code error) { + this->connected = true; + if (error) + throw boost::system::system_error{error}; + }); + } + + ~io_thread() noexcept + { + io_service.stop(); + if (io.joinable()) + io.join(); + } + }; + + struct checked_client + { + std::atomic<bool>* called_; + bool expected_; + + void operator()(boost::system::error_code error, net::socks::client::stream_type::socket&&) const + { + EXPECT_EQ(expected_, bool(error)) << "Socks server: " << error.message(); + ASSERT_TRUE(called_ != nullptr); + (*called_) = true; + } + }; +} + +TEST(socks_client, unsupported_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_connect_command("example.com", 8080)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_resolve_command("example.com")); + EXPECT_TRUE(test_client->buffer().empty()); +} + +TEST(socks_client, no_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_FALSE(net::socks::client::send(std::move(test_client))); +} + +TEST(socks_client, connect_command) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, checked_client{std::addressof(called), false} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_connect_command("example.com", 8080)); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x1f, 0x90, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, connect_command_failed) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, checked_client{std::addressof(called), true} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE( + test_client->set_connect_command( + epee::net_utils::ipv4_network_address{boost::endian::native_to_big(std::uint32_t(5000)), 3000} + ) + ); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x0b, 0xb8, 0x00, 0x00, 0x13, 0x88, 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 91, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, resolve_command) +{ + static std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0xff, 0, 0xad, 0}; + + struct resolve_client : net::socks::client + { + std::atomic<unsigned> called_; + bool expected_; + + resolve_client(stream_type::socket&& proxy) + : net::socks::client(std::move(proxy), net::socks::version::v4a_tor) + , called_(0) + , expected_(false) + {}; + + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) override + { + EXPECT_EQ(this, self.get()); + EXPECT_EQ(expected_, bool(error)) << "Resolve failure: " << error.message(); + + if (!error) + { + ASSERT_EQ(sizeof(reply_bytes), buffer().size()); + EXPECT_EQ(0u, std::memcmp(buffer().data(), reply_bytes, sizeof(reply_bytes))); + } + + ++called_; + } + }; + + io_thread io{}; + stream_type::socket client{io.io_service}; + + auto test_client = std::make_shared<resolve_client>(std::move(client)); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(test_client, io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 0); + + test_client->expected_ = true; + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::send(test_client)); + + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + reply_bytes[1] = 91; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 1); +} + + diff --git a/tests/unit_tests/notify.cpp b/tests/unit_tests/notify.cpp index ceeba8649..4de4a8d0a 100644 --- a/tests/unit_tests/notify.cpp +++ b/tests/unit_tests/notify.cpp @@ -82,7 +82,6 @@ TEST(notify, works) ok = true; break; } - std::cout << "got: [" << s << "]" << std::endl; } } boost::filesystem::remove(name_template); diff --git a/tests/unit_tests/output_distribution.cpp b/tests/unit_tests/output_distribution.cpp index 649752ac7..45f2c135b 100644 --- a/tests/unit_tests/output_distribution.cpp +++ b/tests/unit_tests/output_distribution.cpp @@ -33,7 +33,7 @@ #include "cryptonote_core/cryptonote_core.h" #include "cryptonote_core/tx_pool.h" #include "cryptonote_core/blockchain.h" -#include "testdb.h" +#include "blockchain_db/testdb.h" static const uint64_t test_distribution[32] = { 0, 0, 0, 0, 0, 1, 5, 1, 4, 0, 0, 1, 0, 1, 2, 3, 1, 0, 2, 0, 1, 3, 8, 1, 3, 5, 7, 1, 5, 0, 2, 3 @@ -43,7 +43,7 @@ static const size_t test_distribution_size = sizeof(test_distribution) / sizeof( namespace { -class TestDB: public BaseTestDB +class TestDB: public cryptonote::BaseTestDB { public: TestDB(size_t bc_height = test_distribution_size): blockchain_height(bc_height) { m_open = true; } diff --git a/tests/unit_tests/test_peerlist.cpp b/tests/unit_tests/test_peerlist.cpp index f638c6251..03aa48ea0 100644 --- a/tests/unit_tests/test_peerlist.cpp +++ b/tests/unit_tests/test_peerlist.cpp @@ -37,7 +37,7 @@ TEST(peer_list, peer_list_general) { nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); #define MAKE_IPV4_ADDRESS(a,b,c,d,e) epee::net_utils::ipv4_network_address{MAKE_IP(a,b,c,d),e} #define ADD_GRAY_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple; ple.last_seen=last_seen_;ple.adr = addr_; ple.id = id_;plm.append_with_peer_gray(ple);} #define ADD_WHITE_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple;ple.last_seen=last_seen_; ple.adr = addr_; ple.id = id_;plm.append_with_peer_white(ple);} @@ -77,9 +77,180 @@ TEST(peer_list, merge_peer_lists) //([^ \t]*)\t([^ \t]*):([^ \t]*) \tlast_seen: d(\d+)\.h(\d+)\.m(\d+)\.s(\d+)\n //ADD_NODE_TO_PL("\2", \3, 0x\1, (1353346618 -(\4*60*60*24+\5*60*60+\6*60+\7 )));\n nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); std::vector<nodetool::peerlist_entry> outer_bs; #define ADD_NODE_TO_PL(ip_, port_, id_, timestamp_) { nodetool::peerlist_entry ple; epee::string_tools::get_ip_int32_from_string(ple.adr.ip, ip_); ple.last_seen = timestamp_; ple.adr.port = port_; ple.id = id_;outer_bs.push_back(ple);} +} + +namespace +{ + bool check_empty(nodetool::peerlist_storage& peers, std::initializer_list<epee::net_utils::zone> zones) + { + bool pass = false; + for (const epee::net_utils::zone zone : zones) + { + const nodetool::peerlist_types types{peers.take_zone(zone)}; + EXPECT_TRUE(types.white.empty()); + EXPECT_TRUE(types.gray.empty()); + EXPECT_TRUE(types.anchor.empty()); + pass = (types.white.empty() && types.gray.empty() && types.anchor.empty()); + } + return pass; + } +} + +TEST(peerlist_storage, store) +{ + + using address_type = epee::net_utils::address_type; + using zone = epee::net_utils::zone; + + nodetool::peerlist_storage peers{}; + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + + std::string buffer{}; + { + nodetool::peerlist_types types{}; + types.white.push_back({epee::net_utils::ipv4_network_address{1000, 10}, 44, 55}); + types.white.push_back({net::tor_address::unknown(), 64, 75}); + types.gray.push_back({net::tor_address::unknown(), 99, 88}); + types.gray.push_back({epee::net_utils::ipv4_network_address{2000, 20}, 84, 45}); + types.anchor.push_back({epee::net_utils::ipv4_network_address{999, 654}, 444, 555}); + types.anchor.push_back({net::tor_address::unknown(), 14, 33}); + types.anchor.push_back({net::tor_address::unknown(), 24, 22}); + + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + nodetool::peerlist_types types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + { + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); + + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); } |