diff options
author | Lee Clagett <code@leeclagett.com> | 2018-12-16 17:57:44 +0000 |
---|---|---|
committer | Lee Clagett <code@leeclagett.com> | 2019-01-28 23:56:33 +0000 |
commit | 973403bc9f54ab0722b67a3c76ab6e7bafbfeedc (patch) | |
tree | 01f74938dc99a56c5d20840baa9bce66142847ae | |
parent | Merge pull request #5062 (diff) | |
download | monero-973403bc9f54ab0722b67a3c76ab6e7bafbfeedc.tar.xz |
Adding initial support for broadcasting transactions over Tor
- Support for ".onion" in --add-exclusive-node and --add-peer
- Add --anonymizing-proxy for outbound Tor connections
- Add --anonymous-inbounds for inbound Tor connections
- Support for sharing ".onion" addresses over Tor connections
- Support for broadcasting transactions received over RPC exclusively
over Tor (else broadcast over public IP when Tor not enabled).
Diffstat (limited to '')
39 files changed, 4293 insertions, 826 deletions
diff --git a/ANONYMITY_NETWORKS.md b/ANONYMITY_NETWORKS.md new file mode 100644 index 000000000..6ac8cd999 --- /dev/null +++ b/ANONYMITY_NETWORKS.md @@ -0,0 +1,179 @@ +# Anonymity Networks with Monero + +Currently only Tor has been integrated into Monero. Providing support for +Kovri/I2P should be minimal, but has not yet been attempted. The usage of +these networks is still considered experimental - there are a few pessimistic +cases where privacy is leaked. The design is intended to maximize privacy of +the source of a transaction by broadcasting it over an anonymity network, while +relying on IPv4 for the remainder of messages to make surrounding node attacks +(via sybil) more difficult. + + +## Behavior + +If _any_ anonymity network is enabled, transactions being broadcast that lack +a valid "context" (i.e. the transaction did not come from a p2p connection), +will only be sent to peers on anonymity networks. If an anonymity network is +enabled but no peers over an anonymity network are available, an error is +logged and the transaction is kept for future broadcasting over an anonymity +network. The transaction will not be broadcast unless an anonymity connection +is made or until `monerod` is shutdown and restarted with only public +connections enabled. + + +## P2P Commands + +Only handshakes, peer timed syncs, and transaction broadcast messages are +supported over anonymity networks. If one `--add-exclusive-node` onion address +is specified, then no syncing will take place and only transaction broadcasting +can occur. It is therefore recommended that `--add-exclusive-node` be combined +with additional exclusive IPv4 address(es). + + +## Usage + +Anonymity networks have no seed nodes (the feature is still considered +experimental), so a user must specify an address. If configured properly, +additional peers can be found through typical p2p peerlist sharing. + +### Outbound Connections + +Connecting to an anonymous address requires the command line option +`--proxy` which tells `monerod` the ip/port of a socks proxy provided by a +separate process. On most systems the configuration will look like: + +> `--proxy tor,127.0.0.1:9050,10` +> `--proxy i2p,127.0.0.1:9000` + +which tells `monerod` that ".onion" p2p addresses can be forwarded to a socks +proxy at IP 127.0.0.1 port 9050 with a max of 10 outgoing connections and +".i2p" p2p addresses can be forwarded to a socks proxy at IP 127.0.0.1 port 9000 +with the default max outgoing connections. Since there are no seed nodes for +anonymity connections, peers must be manually specified: + +> `--add-exclusive-node rveahdfho7wo4b2m.onion:28083` +> `--add-peer rveahdfho7wo4b2m.onion:28083` + +Either option can be listed multiple times, and can specify any mix of Tor, +I2P, and IPv4 addresses. Using `--add-exclusive-node` will prevent the usage of +seed nodes on ALL networks, which will typically be undesireable. + +### Inbound Connections + +Receiving anonymity connections is done through the option +`--anonymous-inbound`. This option tells `monerod` the inbound address, network +type, and max connections: + +> `--anonymous-inbound rveahdfho7wo4b2m.onion:28083,127.0.0.28083,25` +> `--anonymous-inbound foobar.i2p:5000,127.0.0.1:30000` + +which tells `monerod` that a max of 25 inbound Tor connections are being +received at address "rveahdfho7wo4b2m.onion:28083" and forwarded to `monerod` +localhost port 28083, and a default max I2P connections are being received at +address "foobar.i2p:5000" and forwarded to `monerod` localhost port 30000. +These addresses will be shared with outgoing peers, over the same network type, +otherwise the peer will not be notified of the peer address by the proxy. + +### Network Types + +#### Tor + +Options `--add-exclusive-node` and `--add-peer` recognize ".onion" addresses, +and will properly forward those addresses to the proxy provided with +`--proxy tor,...`. + +Option `--anonymous-inbound` also recognizes ".onion" addresses, and will +automatically be sent out to outgoing Tor connections so the peer can +distribute the address to its other peers. + +##### Configuration + +Tor must be configured for hidden services. An example configuration ("torrc") +might look like: + +> HiddenServiceDir /var/lib/tor/data/monero +> HiddenServicePort 28083 127.0.0.1:28083 + +This will store key information in `/var/lib/tor/data/monero` and will forward +"Tor port" 28083 to port 28083 of ip 127.0.0.1. The file +`/usr/lib/tor/data/monero/hostname` will contain the ".onion" address for use +with `--anonymous-inbound`. + +#### Kovri/I2P + +Support for this network has not been implemented. Using ".i2p" addresses or +specifying "i2p" will currently generate an error. + + +## Privacy Limitations + +There are currently some techniques that could be used to _possibly_ identify +the machine that broadcast a transaction over an anonymity network. + +### Timestamps + +The peer timed sync command sends the current time in the message. This value +can be used to link an onion address to an IPv4/IPv6 address. If a peer first +sees a transaction over Tor, it could _assume_ (possibly incorrectly) that the +transaction originated from the peer. If both the Tor connection and an +IPv4/IPv6 connection have timestamps that are approximately close in value they +could be used to link the two connections. This is less likely to happen if the +system clock is fairly accurate - many peers on the Monero network should have +similar timestamps. + +#### Mitigation + +Keep the system clock accurate so that fingerprinting is more difficult. In +the future a random offset might be applied to anonymity networks so that if +the system clock is noticeably off (and therefore more fingerprintable), +linking the public IPv4/IPv6 connections with the anonymity networks will be +more difficult. + +### Bandwidth Usage + +An ISP can passively monitor `monerod` connections from a node and observe when +a transaction is sent over a Tor/Kovri connection via timing analysis + size of +data sent during that timeframe. Kovri should provide better protection against +this attack - its connections are not circuit based. However, if a node is +only using Kovri for broadcasting Monero transactions, the total aggregate of +Kovri/I2P data would also leak information. + +#### Mitigation + +There is no current mitigation for the user right now. This attack is fairly +sophisticated, and likely requires support from the internet host of a Monero +user. + +In the near future, "whitening" the amount of data sent over anonymity network +connections will be performed. An attempt will be made to make a transaction +broadcast indistinguishable from a peer timed sync command. + +### Intermittent Monero Syncing + +If a user only runs `monerod` to send a transaction then quit, this can also +be used by an ISP to link a user to a transaction. + +#### Mitigation + +Run `monerod` as often as possible to conceal when transactions are being sent. +Future versions will also have peers that first receive a transaction over an +anonymity network delay the broadcast to public peers by a randomized amount. +This will not completetely mitigate a user who syncs up sends then quits, in +part because this rule is not enforceable, so this mitigation strategy is +simply a best effort attempt. + +### Active Bandwidth Shaping + +An attacker could attempt to bandwidth shape traffic in an attempt to determine +the source of a Tor/Kovri/I2P connection. There isn't great mitigation against +this, but Kovri/I2P should provide better protection against this attack since +the connections are not circuit based. + +#### Mitigation + +The best mitigiation is to use Kovri/I2P instead of Tor. However, Kovri/I2P +has a smaller set of users (less cover traffic) and academic reviews, so there +is a tradeoff in potential isses. Also, anyone attempting this strategy really +wants to uncover a user, it seems unlikely that this would be performed against +every Tor/Kovri/I2P user. + @@ -615,6 +615,12 @@ See [README.i18n.md](README.i18n.md). ## Using Tor +> There is a new, still experimental, [integration with Tor](ANONYMITY_NETWORKS.md). The +> feature allows connecting over IPv4 and Tor simulatenously - IPv4 is used for +> relaying blocks and relaying transactions received by peers whereas Tor is +> used solely for relaying transactions received over local RPC. This provides +> privacy and better protection against surrounding node (sybil) attacks. + While Monero isn't made to integrate with Tor, it can be used wrapped with torsocks, by setting the following configuration parameters and environment variables: diff --git a/contrib/epee/include/net/abstract_tcp_server2.h b/contrib/epee/include/net/abstract_tcp_server2.h index e6b2755af..37f4c782d 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.h +++ b/contrib/epee/include/net/abstract_tcp_server2.h @@ -41,6 +41,7 @@ #include <boost/noncopyable.hpp> #include <boost/shared_ptr.hpp> #include <atomic> +#include <cassert> #include <map> #include <memory> @@ -87,14 +88,25 @@ namespace net_utils { public: typedef typename t_protocol_handler::connection_context t_connection_context; + + struct shared_state : socket_stats + { + shared_state() + : socket_stats(), pfilter(nullptr), config() + {} + + i_connection_filter* pfilter; + typename t_protocol_handler::config_type config; + }; + /// Construct a connection with the given io_service. - explicit connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter * &pfilter - ,t_connection_type connection_type); + boost::shared_ptr<shared_state> state, + t_connection_type connection_type); + + explicit connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type); virtual ~connection() noexcept(false); /// Get the socket associated with the connection. @@ -103,6 +115,9 @@ namespace net_utils /// Start the first asynchronous operation for the connection. bool start(bool is_income, bool is_multithreaded); + // `real_remote` is the actual endpoint (if connection is to proxy, etc.) + bool start(bool is_income, bool is_multithreaded, network_address real_remote); + void get_context(t_connection_context& context_){context_ = context;} void call_back_starter(); @@ -148,7 +163,6 @@ namespace net_utils //boost::array<char, 1024> buffer_; t_connection_context context; - i_connection_filter* &m_pfilter; // TODO what do they mean about wait on destructor?? --rfree : //this should be the last one, because it could be wait on destructor, while other activities possible on other threads @@ -210,7 +224,9 @@ namespace net_utils /// Stop the server. void send_stop_signal(); - bool is_stop_signal_sent(); + bool is_stop_signal_sent() const noexcept { return m_stop_signal_sent; }; + + const std::atomic<bool>& get_stop_signal() const noexcept { return m_stop_signal_sent; } void set_threads_prefix(const std::string& prefix_name); @@ -220,17 +236,28 @@ namespace net_utils void set_connection_filter(i_connection_filter* pfilter); + void set_default_remote(epee::net_utils::network_address remote) + { + default_remote = std::move(remote); + } + + bool add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote); bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0"); template<class t_callback> bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0"); - typename t_protocol_handler::config_type& get_config_object(){return m_config;} + typename t_protocol_handler::config_type& get_config_object() + { + assert(m_state != nullptr); // always set in constructor + return m_state->config; + } int get_binded_port(){return m_port;} long get_connections_count() const { - auto connections_count = (m_sock_count > 0) ? (m_sock_count - 1) : 0; // Socket count minus listening socket + assert(m_state != nullptr); // always set in constructor + auto connections_count = m_state->sock_count > 0 ? (m_state->sock_count - 1) : 0; // Socket count minus listening socket return connections_count; } @@ -292,9 +319,6 @@ namespace net_utils return true; } - protected: - typename t_protocol_handler::config_type m_config; - private: /// Run the server's io_service loop. bool worker_thread(); @@ -303,21 +327,21 @@ namespace net_utils bool is_thread_worker(); + const boost::shared_ptr<typename connection<t_protocol_handler>::shared_state> m_state; + /// The io_service used to perform asynchronous operations. std::unique_ptr<boost::asio::io_service> m_io_service_local_instance; boost::asio::io_service& io_service_; /// Acceptor used to listen for incoming connections. boost::asio::ip::tcp::acceptor acceptor_; + epee::net_utils::network_address default_remote; std::atomic<bool> m_stop_signal_sent; uint32_t m_port; - std::atomic<long> m_sock_count; - std::atomic<long> m_sock_number; std::string m_address; std::string m_thread_name_prefix; //TODO: change to enum server_type, now used size_t m_threads_count; - i_connection_filter* m_pfilter; std::vector<boost::shared_ptr<boost::thread> > m_threads; boost::thread::id m_main_thread_id; critical_section m_threads_lock; diff --git a/contrib/epee/include/net/abstract_tcp_server2.inl b/contrib/epee/include/net/abstract_tcp_server2.inl index 457ee2dd4..9c89a18cf 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.inl +++ b/contrib/epee/include/net/abstract_tcp_server2.inl @@ -40,6 +40,7 @@ #include <boost/asio/deadline_timer.hpp> #include <boost/date_time/posix_time/posix_time.hpp> // TODO #include <boost/thread/condition_variable.hpp> // TODO +#include <boost/make_shared.hpp> #include "warnings.h" #include "string_tools.h" #include "misc_language.h" @@ -62,6 +63,13 @@ namespace epee { namespace net_utils { + template<typename T> + T& check_and_get(boost::shared_ptr<T>& ptr) + { + CHECK_AND_ASSERT_THROW_MES(bool(ptr), "shared_state cannot be null"); + return *ptr; + } + /************************************************************************/ /* */ /************************************************************************/ @@ -69,25 +77,31 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> connection<t_protocol_handler>::connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter* &pfilter - ,t_connection_type connection_type + boost::shared_ptr<shared_state> state, + t_connection_type connection_type + ) + : connection(boost::asio::ip::tcp::socket{io_service}, std::move(state), connection_type) + { + } + + template<class t_protocol_handler> + connection<t_protocol_handler>::connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type ) : - connection_basic(io_service, ref_sock_count, sock_number), - m_protocol_handler(this, config, context), - m_pfilter( pfilter ), + connection_basic(std::move(sock), state), + m_protocol_handler(this, check_and_get(state).config, context), m_connection_type( connection_type ), m_throttle_speed_in("speed_in", "throttle_speed_in"), m_throttle_speed_out("speed_out", "throttle_speed_out"), - m_timer(io_service), + m_timer(socket_.get_io_service()), m_local(false), m_ready_to_close(false) { MDEBUG("test, connection constructor set m_connection_type="<<m_connection_type); } + PRAGMA_WARNING_DISABLE_VS(4355) //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -127,34 +141,44 @@ PRAGMA_WARNING_DISABLE_VS(4355) { TRY_ENTRY(); + boost::system::error_code ec; + auto remote_ep = socket_.remote_endpoint(ec); + CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); + CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); + + const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; + return start(is_income, is_multithreaded, ipv4_network_address{uint32_t(ip_), remote_ep.port()}); + CATCH_ENTRY_L0("connection<t_protocol_handler>::start()", false); + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> + bool connection<t_protocol_handler>::start(bool is_income, bool is_multithreaded, network_address real_remote) + { + TRY_ENTRY(); + // Use safe_shared_from_this, because of this is public method and it can be called on the object being deleted auto self = safe_shared_from_this(); if(!self) return false; m_is_multithreaded = is_multithreaded; + m_local = real_remote.is_loopback() || real_remote.is_local(); - boost::system::error_code ec; - auto remote_ep = socket_.remote_endpoint(ec); - CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); - CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); + // create a random uuid, we don't need crypto strength here + const boost::uuids::uuid random_uuid = boost::uuids::random_generator()(); + + context = t_connection_context{}; + context.set_details(random_uuid, std::move(real_remote), is_income); + boost::system::error_code ec; auto local_ep = socket_.local_endpoint(ec); CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get local endpoint: " << ec.message() << ':' << ec.value()); - context = boost::value_initialized<t_connection_context>(); - const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; - m_local = epee::net_utils::is_ip_loopback(ip_) || epee::net_utils::is_ip_local(ip_); - - // create a random uuid, we don't need crypto strength here - const boost::uuids::uuid random_uuid = boost::uuids::random_generator()(); - - context.set_details(random_uuid, epee::net_utils::ipv4_network_address(ip_, remote_ep.port()), is_income); _dbg3("[sock " << socket_.native_handle() << "] new connection from " << print_connection_context_short(context) << " to " << local_ep.address().to_string() << ':' << local_ep.port() << - ", total sockets objects " << m_ref_sock_count); + ", total sockets objects " << get_stats().sock_count); - if(m_pfilter && !m_pfilter->is_remote_host_allowed(context.m_remote_address)) + if(static_cast<shared_state&>(get_stats()).pfilter && !static_cast<shared_state&>(get_stats()).pfilter->is_remote_host_allowed(context.m_remote_address)) { _dbg2("[sock " << socket_.native_handle() << "] host denied " << context.m_remote_address.host_str() << ", shutdowning connection"); close(); @@ -279,7 +303,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) } MDEBUG(" connection type " << to_string( m_connection_type ) << " " << socket_.local_endpoint().address().to_string() << ":" << socket_.local_endpoint().port() - << " <--> " << address << ":" << port); + << " <--> " << context.m_remote_address.str() << " (via " << address << ":" << port << ")"); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -784,12 +808,14 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server( t_connection_type connection_type ) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), m_io_service_local_instance(new boost::asio::io_service()), io_service_(*m_io_service_local_instance.get()), acceptor_(io_service_), + default_remote(), m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + m_threads_count(0), + m_thread_index(0), m_connection_type( connection_type ), new_connection_() { @@ -799,11 +825,13 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server(boost::asio::io_service& extarnal_io_service, t_connection_type connection_type) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), io_service_(extarnal_io_service), acceptor_(io_service_), - m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + default_remote(), + m_stop_signal_sent(false), m_port(0), + m_threads_count(0), + m_thread_index(0), m_connection_type(connection_type), new_connection_() { @@ -844,7 +872,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) boost::asio::ip::tcp::endpoint binded_endpoint = acceptor_.local_endpoint(); m_port = binded_endpoint.port(); MDEBUG("start accept"); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -922,7 +950,8 @@ POP_WARNINGS template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::set_connection_filter(i_connection_filter* pfilter) { - m_pfilter = pfilter; + assert(m_state != nullptr); // always set in constructor + m_state->pfilter = pfilter; } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -1030,12 +1059,6 @@ POP_WARNINGS } //--------------------------------------------------------------------------------- template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::is_stop_signal_sent() - { - return m_stop_signal_sent; - } - //--------------------------------------------------------------------------------- - template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::handle_accept(const boost::system::error_code& e) { MDEBUG("handle_accept"); @@ -1048,7 +1071,7 @@ POP_WARNINGS new_connection_->setRpcStation(); // hopefully this is not needed actually } connection_ptr conn(std::move(new_connection_)); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -1056,7 +1079,10 @@ POP_WARNINGS boost::asio::socket_base::keep_alive opt(true); conn->socket().set_option(opt); - conn->start(true, 1 < m_threads_count); + if (default_remote.get_type_id() == net_utils::address_type::invalid) + conn->start(true, 1 < m_threads_count); + else + conn->start(true, 1 < m_threads_count, default_remote); conn->save_dbg_log(); return; } @@ -1071,20 +1097,41 @@ POP_WARNINGS } // error path, if e or exception - _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_state->sock_count); misc_utils::sleep_no_w(100); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> + bool boosted_tcp_server<t_protocol_handler>::add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote) + { + if(std::addressof(get_io_service()) == std::addressof(sock.get_io_service())) + { + connection_ptr conn(new connection<t_protocol_handler>(std::move(sock), m_state, m_connection_type)); + if(conn->start(false, 1 < m_threads_count, std::move(real_remote))) + { + conn->get_context(out); + conn->save_dbg_log(); + return true; + } + } + else + { + MWARNING(out << " was not added, socket/io_service mismatch"); + } + return false; + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip) { TRY_ENTRY(); - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) ); connections_mutex.lock(); connections_.insert(new_connection_l); MDEBUG("connections_ size now " << connections_.size()); @@ -1187,7 +1234,8 @@ POP_WARNINGS } else { - _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_state->sock_count); } new_connection_l->save_dbg_log(); @@ -1201,7 +1249,7 @@ POP_WARNINGS bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip) { TRY_ENTRY(); - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) ); connections_mutex.lock(); connections_.insert(new_connection_l); MDEBUG("connections_ size now " << connections_.size()); diff --git a/contrib/epee/include/net/connection_basic.hpp b/contrib/epee/include/net/connection_basic.hpp index 9b6fc14a7..b1b271db9 100644 --- a/contrib/epee/include/net/connection_basic.hpp +++ b/contrib/epee/include/net/connection_basic.hpp @@ -55,6 +55,15 @@ namespace epee { namespace net_utils { + struct socket_stats + { + socket_stats() + : sock_count(0), sock_number(0) + {} + + std::atomic<long> sock_count; + std::atomic<long> sock_number; + }; /************************************************************************/ /* */ @@ -72,6 +81,8 @@ class connection_basic_pimpl; // PIMPL for this class std::string to_string(t_connection_type type); class connection_basic { // not-templated base class for rapid developmet of some code parts + // beware of removing const, net_utils::connection is sketchily doing a cast to prevent storing ptr twice + const boost::shared_ptr<socket_stats> m_stats; public: std::unique_ptr< connection_basic_pimpl > mI; // my Implementation @@ -86,13 +97,15 @@ class connection_basic { // not-templated base class for rapid developmet of som /// Socket for the connection. boost::asio::ip::tcp::socket socket_; - std::atomic<long> &m_ref_sock_count; // reference to external counter of existing sockets that we will ++/-- public: // first counter is the ++/-- count of current sockets, the other socket_number is only-increasing ++ number generator - connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number); + connection_basic(boost::asio::ip::tcp::socket&& socket, boost::shared_ptr<socket_stats> stats); virtual ~connection_basic() noexcept(false); + //! \return `socket_stats` object passed in construction (ptr never changes). + socket_stats& get_stats() noexcept { return *m_stats; /* verified in constructor */ } + // various handlers to be called from connection class: void do_send_handler_write(const void * ptr , size_t cb); void do_send_handler_write_from_queue(const boost::system::error_code& e, size_t cb , int q_len); // from handle_write, sending next part diff --git a/contrib/epee/include/net/enums.h b/contrib/epee/include/net/enums.h new file mode 100644 index 000000000..078a4b274 --- /dev/null +++ b/contrib/epee/include/net/enums.h @@ -0,0 +1,65 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +namespace epee +{ +namespace net_utils +{ + enum class address_type : std::uint8_t + { + // Do not change values, this will break serialization + invalid = 0, + ipv4 = 1, + ipv6 = 2, + i2p = 3, + tor = 4 + }; + + enum class zone : std::uint8_t + { + invalid = 0, + public_ = 1, // public is keyword + i2p = 2, + tor = 3 + }; + + // implementations in src/net_utils_base.cpp + + //! \return String name of zone or "invalid" on error. + const char* zone_to_string(zone value) noexcept; + + //! \return `zone` enum of `value` or `zone::invalid` on error. + zone zone_from_string(boost::string_ref value) noexcept; +} // net_utils +} // epee + diff --git a/contrib/epee/include/net/net_utils_base.h b/contrib/epee/include/net/net_utils_base.h index a9e458626..82f8a7fe8 100644 --- a/contrib/epee/include/net/net_utils_base.h +++ b/contrib/epee/include/net/net_utils_base.h @@ -33,6 +33,7 @@ #include <boost/asio/io_service.hpp> #include <typeinfo> #include <type_traits> +#include "enums.h" #include "serialization/keyvalue_serialization.h" #include "misc_log_ex.h" @@ -43,6 +44,11 @@ #define MAKE_IP( a1, a2, a3, a4 ) (a1|(a2<<8)|(a3<<16)|(a4<<24)) #endif +namespace net +{ + class tor_address; +} + namespace epee { namespace net_utils @@ -53,6 +59,10 @@ namespace net_utils uint16_t m_port; public: + constexpr ipv4_network_address() noexcept + : ipv4_network_address(0, 0) + {} + constexpr ipv4_network_address(uint32_t ip, uint16_t port) noexcept : m_ip(ip), m_port(port) {} @@ -67,9 +77,10 @@ namespace net_utils std::string host_str() const; bool is_loopback() const; bool is_local() const; - static constexpr uint8_t get_type_id() noexcept { return ID; } + static constexpr address_type get_type_id() noexcept { return address_type::ipv4; } + static constexpr zone get_zone() noexcept { return zone::public_; } + static constexpr bool is_blockable() noexcept { return true; } - static const uint8_t ID = 1; BEGIN_KV_SERIALIZE_MAP() KV_SERIALIZE(m_ip) KV_SERIALIZE(m_port) @@ -103,7 +114,9 @@ namespace net_utils virtual std::string host_str() const = 0; virtual bool is_loopback() const = 0; virtual bool is_local() const = 0; - virtual uint8_t get_type_id() const = 0; + virtual address_type get_type_id() const = 0; + virtual zone get_zone() const = 0; + virtual bool is_blockable() const = 0; }; template<typename T> @@ -131,7 +144,9 @@ namespace net_utils virtual std::string host_str() const override { return value.host_str(); } virtual bool is_loopback() const override { return value.is_loopback(); } virtual bool is_local() const override { return value.is_local(); } - virtual uint8_t get_type_id() const override { return value.get_type_id(); } + virtual address_type get_type_id() const override { return value.get_type_id(); } + virtual zone get_zone() const override { return value.get_zone(); } + virtual bool is_blockable() const override { return value.is_blockable(); } }; std::shared_ptr<interface> self; @@ -146,6 +161,23 @@ namespace net_utils throw std::bad_cast{}; return static_cast<implementation<Type_>*>(self_)->value; } + + template<typename T, typename t_storage> + bool serialize_addr(std::false_type, t_storage& stg, typename t_storage::hsection hparent) + { + T addr{}; + if (!epee::serialization::selector<false>::serialize(addr, stg, hparent, "addr")) + return false; + *this = std::move(addr); + return true; + } + + template<typename T, typename t_storage> + bool serialize_addr(std::true_type, t_storage& stg, typename t_storage::hsection hparent) const + { + return epee::serialization::selector<true>::serialize(as<T>(), stg, hparent, "addr"); + } + public: network_address() : self(nullptr) {} template<typename T> @@ -158,43 +190,32 @@ namespace net_utils std::string host_str() const { return self ? self->host_str() : "<none>"; } bool is_loopback() const { return self ? self->is_loopback() : false; } bool is_local() const { return self ? self->is_local() : false; } - uint8_t get_type_id() const { return self ? self->get_type_id() : 0; } + address_type get_type_id() const { return self ? self->get_type_id() : address_type::invalid; } + zone get_zone() const { return self ? self->get_zone() : zone::invalid; } + bool is_blockable() const { return self ? self->is_blockable() : false; } template<typename Type> const Type &as() const { return as_mutable<const Type>(); } BEGIN_KV_SERIALIZE_MAP() - uint8_t type = is_store ? this_ref.get_type_id() : 0; + // need to `#include "net/tor_address.h"` when serializing `network_address` + static constexpr std::integral_constant<bool, is_store> is_store_{}; + + std::uint8_t type = std::uint8_t(is_store ? this_ref.get_type_id() : address_type::invalid); if (!epee::serialization::selector<is_store>::serialize(type, stg, hparent_section, "type")) return false; - switch (type) + + switch (address_type(type)) { - case ipv4_network_address::ID: - { - if (!is_store) - { - const_cast<network_address&>(this_ref) = ipv4_network_address{0, 0}; - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - MDEBUG("Found as addr: " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as<ipv4_network_address>()")) - MDEBUG("Found as template as<ipv4_network_address>(): " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as_mutable<ipv4_network_address>()")) - MDEBUG("Found as template as_mutable<ipv4_network_address>(): " << this_ref.str()); - else - { - MWARNING("Address not found"); - return false; - } - } - else - { - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (!epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - return false; - } + case address_type::ipv4: + return this_ref.template serialize_addr<ipv4_network_address>(is_store_, stg, hparent_section); + case address_type::tor: + return this_ref.template serialize_addr<net::tor_address>(is_store_, stg, hparent_section); + case address_type::invalid: + default: break; - } - default: MERROR("Unsupported network address type: " << (unsigned)type); return false; } + + MERROR("Unsupported network address type: " << (unsigned)type); + return false; END_KV_SERIALIZE_MAP() }; @@ -211,8 +232,6 @@ namespace net_utils inline bool operator>=(const network_address& lhs, const network_address& rhs) { return !lhs.less(rhs); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port = 0); - /************************************************************************/ /* */ /************************************************************************/ @@ -250,7 +269,7 @@ namespace net_utils {} connection_context_base(): m_connection_id(), - m_remote_address(ipv4_network_address{0,0}), + m_remote_address(), m_is_income(false), m_started(time(NULL)), m_last_recv(0), diff --git a/contrib/epee/src/connection_basic.cpp b/contrib/epee/src/connection_basic.cpp index 7d145ee46..f5f9b59fe 100644 --- a/contrib/epee/src/connection_basic.cpp +++ b/contrib/epee/src/connection_basic.cpp @@ -103,7 +103,7 @@ namespace net_utils // connection_basic_pimpl // ================================================================================================ -connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name) { } +connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name), m_peer_number(0) { } // ================================================================================================ // connection_basic @@ -113,27 +113,31 @@ connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_thro int connection_basic_pimpl::m_default_tos; // methods: -connection_basic::connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number) - : +connection_basic::connection_basic(boost::asio::ip::tcp::socket&& socket, boost::shared_ptr<socket_stats> stats) + : + m_stats(std::move(stats)), mI( new connection_basic_pimpl("peer") ), - strand_(io_service), - socket_(io_service), + strand_(socket.get_io_service()), + socket_(std::move(socket)), m_want_close_connection(false), - m_was_shutdown(false), - m_ref_sock_count(ref_sock_count) -{ - ++ref_sock_count; // increase the global counter - mI->m_peer_number = sock_number.fetch_add(1); // use, and increase the generated number + m_was_shutdown(false) +{ + // add nullptr checks if removed + CHECK_AND_ASSERT_THROW_MES(bool(m_stats), "stats shared_ptr cannot be null"); + + ++(m_stats->sock_count); // increase the global counter + mI->m_peer_number = m_stats->sock_number.fetch_add(1); // use, and increase the generated number std::string remote_addr_str = "?"; try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; - _note("Spawned connection p2p#"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_ref_sock_count); + _note("Spawned connection p2p#"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_stats->sock_count); } connection_basic::~connection_basic() noexcept(false) { + --(m_stats->sock_count); + std::string remote_addr_str = "?"; - m_ref_sock_count--; try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; _note("Destructing connection p2p#"<<mI->m_peer_number << " to " << remote_addr_str); } diff --git a/contrib/epee/src/net_utils_base.cpp b/contrib/epee/src/net_utils_base.cpp index 263b344b4..9b781027e 100644 --- a/contrib/epee/src/net_utils_base.cpp +++ b/contrib/epee/src/net_utils_base.cpp @@ -8,8 +8,6 @@ namespace epee { namespace net_utils { - const uint8_t ipv4_network_address::ID; - bool ipv4_network_address::equal(const ipv4_network_address& other) const noexcept { return is_same_host(other) && port() == other.port(); } @@ -58,20 +56,6 @@ namespace epee { namespace net_utils return self_->is_same_host(*other_self); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port) - { - uint32_t ip; - uint16_t port; - if (epee::string_tools::parse_peer_from_string(ip, port, string)) - { - if (default_port && !port) - port = default_port; - address = ipv4_network_address{ip, port}; - return true; - } - return false; - } - std::string print_connection_context(const connection_context_base& ctx) { std::stringstream ss; @@ -86,5 +70,31 @@ namespace epee { namespace net_utils return ss.str(); } + const char* zone_to_string(zone value) noexcept + { + switch (value) + { + case zone::public_: + return "public"; + case zone::i2p: + return "i2p"; + case zone::tor: + return "tor"; + default: + break; + } + return "invalid"; + } + + zone zone_from_string(const boost::string_ref value) noexcept + { + if (value == "public") + return zone::public_; + if (value == "i2p") + return zone::i2p; + if (value == "tor") + return zone::tor; + return zone::invalid; + } }} diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index a0b62da77..f1454b48e 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -110,6 +110,7 @@ add_subdirectory(checkpoints) add_subdirectory(cryptonote_basic) add_subdirectory(cryptonote_core) add_subdirectory(multisig) +add_subdirectory(net) if(NOT IOS) add_subdirectory(blockchain_db) endif() diff --git a/src/cryptonote_config.h b/src/cryptonote_config.h index 4f652cd42..93db71705 100644 --- a/src/cryptonote_config.h +++ b/src/cryptonote_config.h @@ -108,6 +108,7 @@ #define P2P_DEFAULT_PACKET_MAX_SIZE 50000000 //50000000 bytes maximum packet size #define P2P_DEFAULT_PEERS_IN_HANDSHAKE 250 #define P2P_DEFAULT_CONNECTION_TIMEOUT 5000 //5 seconds +#define P2P_DEFAULT_TOR_CONNECT_TIMEOUT 20 // seconds #define P2P_DEFAULT_PING_CONNECTION_TIMEOUT 2000 //2 seconds #define P2P_DEFAULT_INVOKE_TIMEOUT 60*2*1000 //2 minutes #define P2P_DEFAULT_HANDSHAKE_INVOKE_TIMEOUT 5000 //5 seconds diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.h b/src/cryptonote_protocol/cryptonote_protocol_handler.h index a1bd9171c..efd986b53 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.h +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.h @@ -173,15 +173,6 @@ namespace cryptonote //handler_response_blocks_now(blob.size()); // XXX return m_p2p->invoke_notify_to_peer(t_parameter::ID, epee::strspan<uint8_t>(blob), context); } - - template<class t_parameter> - bool relay_post_notify(typename t_parameter::request& arg, cryptonote_connection_context& exclude_context) - { - LOG_PRINT_L2("[" << epee::net_utils::print_connection_context_short(exclude_context) << "] post relay " << typeid(t_parameter).name() << " -->"); - std::string arg_buff; - epee::serialization::store_t_to_binary(arg, arg_buff); - return m_p2p->relay_notify_to_all(t_parameter::ID, epee::strspan<uint8_t>(arg_buff), exclude_context); - } }; } // namespace diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.inl b/src/cryptonote_protocol/cryptonote_protocol_handler.inl index 61a211094..c1459cbb6 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.inl +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.inl @@ -226,7 +226,7 @@ namespace cryptonote cnx.host = cntxt.m_remote_address.host_str(); cnx.ip = ""; cnx.port = ""; - if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { cnx.ip = cnx.host; cnx.port = std::to_string(cntxt.m_remote_address.as<epee::net_utils::ipv4_network_address>().port()); @@ -333,6 +333,13 @@ namespace cryptonote return true; } + // No chain synchronization over hidden networks (tor, i2p, etc.) + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + { + context.m_state = cryptonote_connection_context::state_normal; + return true; + } + if (hshd.current_height > target) { /* As I don't know if accessing hshd from core could be a good practice, @@ -2058,20 +2065,20 @@ skip: fluffy_arg.b.txs = fluffy_txs; // sort peers between fluffy ones and others - std::list<boost::uuids::uuid> fullConnections, fluffyConnections; + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> fullConnections, fluffyConnections; m_p2p->for_each_connection([this, &exclude_context, &fullConnections, &fluffyConnections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) { - if (peer_id && exclude_context.m_connection_id != context.m_connection_id) + if (peer_id && exclude_context.m_connection_id != context.m_connection_id && context.m_remote_address.get_zone() == epee::net_utils::zone::public_) { if(m_core.fluffy_blocks_enabled() && (support_flags & P2P_SUPPORT_FLAG_FLUFFY_BLOCKS)) { LOG_DEBUG_CC(context, "PEER SUPPORTS FLUFFY BLOCKS - RELAYING THIN/COMPACT WHATEVER BLOCK"); - fluffyConnections.push_back(context.m_connection_id); + fluffyConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } else { LOG_DEBUG_CC(context, "PEER DOESN'T SUPPORT FLUFFY BLOCKS - RELAYING FULL BLOCK"); - fullConnections.push_back(context.m_connection_id); + fullConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } } return true; @@ -2082,13 +2089,13 @@ skip: { std::string fluffyBlob; epee::serialization::store_t_to_binary(fluffy_arg, fluffyBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), fluffyConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), std::move(fluffyConnections)); } if (!fullConnections.empty()) { std::string fullBlob; epee::serialization::store_t_to_binary(arg, fullBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), fullConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), std::move(fullConnections)); } return true; @@ -2097,6 +2104,12 @@ skip: template<class t_core> bool t_cryptonote_protocol_handler<t_core>::relay_transactions(NOTIFY_NEW_TRANSACTIONS::request& arg, cryptonote_connection_context& exclude_context) { + const bool hide_tx_broadcast = + 1 < m_p2p->get_zone_count() && exclude_context.m_remote_address.get_zone() == epee::net_utils::zone::invalid; + + if (hide_tx_broadcast) + MDEBUG("Attempting to conceal origin of tx via anonymity network connection(s)"); + // no check for success, so tell core they're relayed unconditionally const bool pad_transactions = m_core.pad_transactions(); size_t bytes = pad_transactions ? 9 /* header */ + 4 /* 1 + 'txs' */ + tools::get_varint_data(arg.txs.size()).size() : 0; @@ -2131,7 +2144,30 @@ skip: // if the size of _ moved enough, we might lose byte in size encoding, we don't care } - return relay_post_notify<NOTIFY_NEW_TRANSACTIONS>(arg, exclude_context); + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections; + m_p2p->for_each_connection([hide_tx_broadcast, &exclude_context, &connections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) + { + const epee::net_utils::zone current_zone = context.m_remote_address.get_zone(); + const bool broadcast_to_peer = + peer_id && + (hide_tx_broadcast != bool(current_zone == epee::net_utils::zone::public_)) && + exclude_context.m_connection_id != context.m_connection_id; + + if (broadcast_to_peer) + connections.push_back({current_zone, context.m_connection_id}); + + return true; + }); + + if (connections.empty()) + MERROR("Transaction not relayed - no" << (hide_tx_broadcast ? " privacy": "") << " peers available"); + else + { + std::string fullBlob; + epee::serialization::store_t_to_binary(arg, fullBlob); + m_p2p->relay_notify_to_list(NOTIFY_NEW_TRANSACTIONS::ID, epee::strspan<uint8_t>(fullBlob), std::move(connections)); + } + return true; } //------------------------------------------------------------------------------------------------------------------------ template<class t_core> diff --git a/src/net/CMakeLists.txt b/src/net/CMakeLists.txt new file mode 100644 index 000000000..a81372125 --- /dev/null +++ b/src/net/CMakeLists.txt @@ -0,0 +1,34 @@ +# Copyright (c) 2018, The Monero Project +# +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without modification, are +# permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this list of +# conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, this list +# of conditions and the following disclaimer in the documentation and/or other +# materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its contributors may be +# used to endorse or promote products derived from this software without specific +# prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set(net_sources error.cpp parse.cpp socks.cpp tor_address.cpp) +set(net_headers error.h parse.h socks.h tor_address.h) + +monero_add_library(net ${net_sources} ${net_headers}) +target_link_libraries(net epee ${Boost_ASIO_LIBRARY}) + diff --git a/src/net/error.cpp b/src/net/error.cpp new file mode 100644 index 000000000..037f44d52 --- /dev/null +++ b/src/net/error.cpp @@ -0,0 +1,92 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "error.h" + +#include <string> + +namespace +{ + struct net_category : std::error_category + { + net_category() noexcept + : std::error_category() + {} + + const char* name() const noexcept override + { + return "net::error_category"; + } + + std::string message(int value) const override + { + switch (net::error(value)) + { + case net::error::expected_tld: + return "Expected top-level domain"; + case net::error::invalid_host: + return "Host value is not valid"; + case net::error::invalid_i2p_address: + return "Invalid I2P address"; + case net::error::invalid_port: + return "Invalid port value (expected 0-65535)"; + case net::error::invalid_tor_address: + return "Invalid Tor address"; + case net::error::unsupported_address: + return "Network address not supported"; + default: + break; + } + + return "Unknown net::error"; + } + + std::error_condition default_error_condition(int value) const noexcept override + { + switch (net::error(value)) + { + case net::error::invalid_port: + return std::errc::result_out_of_range; + case net::error::expected_tld: + case net::error::invalid_tor_address: + default: + break; + } + return std::error_condition{value, *this}; + } + }; +} // anonymous + +namespace net +{ + std::error_category const& error_category() noexcept + { + static const net_category instance{}; + return instance; + } +} diff --git a/src/net/error.h b/src/net/error.h new file mode 100644 index 000000000..c8338f7e2 --- /dev/null +++ b/src/net/error.h @@ -0,0 +1,64 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <system_error> +#include <type_traits> + +namespace net +{ + //! General net errors + enum class error : int + { + // 0 reserved for success (as per expect<T>) + expected_tld = 1, //!< Expected a tld + invalid_host, //!< Hostname is not valid + invalid_i2p_address, + invalid_port, //!< Outside of 0-65535 range + invalid_tor_address,//!< Invalid base32 or length + unsupported_address //!< Type not supported by `get_network_address` + }; + + //! \return `std::error_category` for `net` namespace. + std::error_category const& error_category() noexcept; + + //! \return `net::error` as a `std::error_code` value. + inline std::error_code make_error_code(error value) noexcept + { + return std::error_code{int(value), error_category()}; + } +} + +namespace std +{ + template<> + struct is_error_code_enum<::net::error> + : true_type + {}; +} diff --git a/src/net/fwd.h b/src/net/fwd.h new file mode 100644 index 000000000..ee7c539b0 --- /dev/null +++ b/src/net/fwd.h @@ -0,0 +1,45 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> + +namespace net +{ + enum class error : int; + class tor_address; + + namespace socks + { + class client; + template<typename> class connect_handler; + enum class error : int; + enum class version : std::uint8_t; + } +} diff --git a/src/net/parse.cpp b/src/net/parse.cpp new file mode 100644 index 000000000..ebf91eeff --- /dev/null +++ b/src/net/parse.cpp @@ -0,0 +1,60 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "parse.h" + +#include "net/tor_address.h" +#include "string_tools.h" + +namespace net +{ + expect<epee::net_utils::network_address> + get_network_address(const boost::string_ref address, const std::uint16_t default_port) + { + const boost::string_ref host = address.substr(0, address.rfind(':')); + + if (host.empty()) + return make_error_code(net::error::invalid_host); + if (host.ends_with(".onion")) + return tor_address::make(address, default_port); + if (host.ends_with(".i2p")) + return make_error_code(net::error::invalid_i2p_address); // not yet implemented (prevent public DNS lookup) + + std::uint16_t port = default_port; + if (host.size() < address.size()) + { + if (!epee::string_tools::get_xtype_from_string(port, std::string{address.substr(host.size() + 1)})) + return make_error_code(net::error::invalid_port); + } + + std::uint32_t ip = 0; + if (epee::string_tools::get_ip_int32_from_string(ip, std::string{host})) + return {epee::net_utils::ipv4_network_address{ip, port}}; + return make_error_code(net::error::unsupported_address); + } +} diff --git a/src/net/parse.h b/src/net/parse.h new file mode 100644 index 000000000..9195ddc2b --- /dev/null +++ b/src/net/parse.h @@ -0,0 +1,54 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +#include "common/expect.h" +#include "net/net_utils_base.h" + +namespace net +{ + /*! + Identifies onion and IPv4 addresses and returns them as a generic + `network_address`. If the type is unsupported, it might be a hostname, + and `error() == net::error::kUnsupportedAddress` is returned. + + \param address An onion address, ipv4 address or hostname. Hostname + will return an error. + \param default_port If `address` does not specify a port, this value + will be used. + + \return A tor or IPv4 address, else error. + */ + expect<epee::net_utils::network_address> + get_network_address(boost::string_ref address, std::uint16_t default_port); +} + diff --git a/src/net/socks.cpp b/src/net/socks.cpp new file mode 100644 index 000000000..f31efc8c1 --- /dev/null +++ b/src/net/socks.cpp @@ -0,0 +1,308 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "socks.h" + +#include <algorithm> +#include <boost/asio/buffer.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/arithmetic.hpp> +#include <boost/endian/conversion.hpp> +#include <cstring> +#include <limits> +#include <string> + +#include "net/net_utils_base.h" +#include "net/tor_address.h" + +namespace net +{ +namespace socks +{ + namespace + { + constexpr const unsigned v4_reply_size = 8; + constexpr const std::uint8_t v4_connect_command = 1; + constexpr const std::uint8_t v4tor_resolve_command = 0xf0; + constexpr const std::uint8_t v4_request_granted = 90; + + struct v4_header + { + std::uint8_t version; + std::uint8_t command_code; + boost::endian::big_uint16_t port; + boost::endian::big_uint32_t ip; + }; + + std::size_t write_domain_header(epee::span<std::uint8_t> out, const std::uint8_t command, const std::uint16_t port, const boost::string_ref domain) + { + if (std::numeric_limits<std::size_t>::max() - sizeof(v4_header) - 2 < domain.size()) + return 0; + + const std::size_t buf_size = sizeof(v4_header) + domain.size() + 2; + if (out.size() < buf_size) + return 0; + + // version 4, 1 indicates invalid ip for domain extension + const v4_header temp{4, command, port, std::uint32_t(1)}; + std::memcpy(out.data(), std::addressof(temp), sizeof(temp)); + out.remove_prefix(sizeof(temp)); + + *(out.data()) = 0; + out.remove_prefix(1); + + std::memcpy(out.data(), domain.data(), domain.size()); + out.remove_prefix(domain.size()); + + *(out.data()) = 0; + return buf_size; + } + + struct socks_category : boost::system::error_category + { + explicit socks_category() noexcept + : boost::system::error_category() + {} + + const char* name() const noexcept override + { + return "net::socks::error_category"; + } + + virtual std::string message(int value) const override + { + switch (socks::error(value)) + { + case socks::error::rejected: + return "Socks request rejected or failed"; + case socks::error::identd_connection: + return "Socks request rejected because server cannot connect to identd on the client"; + case socks::error::identd_user: + return "Socks request rejected because the client program and identd report different user-ids"; + + case socks::error::bad_read: + return "Socks boost::async_read read fewer bytes than expected"; + case socks::error::bad_write: + return "Socks boost::async_write wrote fewer bytes than expected"; + case socks::error::unexpected_version: + return "Socks server returned unexpected version in reply"; + + default: + break; + } + return "Unknown net::socks::error"; + } + + boost::system::error_condition default_error_condition(int value) const noexcept override + { + switch (socks::error(value)) + { + case socks::error::bad_read: + case socks::error::bad_write: + return boost::system::errc::io_error; + case socks::error::unexpected_version: + return boost::system::errc::protocol_error; + default: + break; + }; + if (1 <= value && value <= 256) + return boost::system::errc::protocol_error; + + return boost::system::error_condition{value, *this}; + } + }; + } + + const boost::system::error_category& error_category() noexcept + { + static const socks_category instance{}; + return instance; + } + + struct client::completed + { + std::shared_ptr<client> self_; + + void operator()(const boost::system::error_code error, const std::size_t bytes) const + { + static_assert(1 < sizeof(self_->buffer_), "buffer too small for v4 response"); + + if (self_) + { + client& self = *self_; + self.buffer_size_ = std::min(bytes, sizeof(self.buffer_)); + + if (error) + self.done(error, std::move(self_)); + else if (self.buffer().size() < sizeof(v4_header)) + self.done(socks::error::bad_read, std::move(self_)); + else if (self.buffer_[0] != 0) // response version + self.done(socks::error::unexpected_version, std::move(self_)); + else if (self.buffer_[1] != v4_request_granted) + self.done(socks::error(int(self.buffer_[1]) + 1), std::move(self_)); + else + self.done(boost::system::error_code{}, std::move(self_)); + } + } + }; + + struct client::read + { + std::shared_ptr<client> self_; + + static boost::asio::mutable_buffers_1 get_buffer(client& self) noexcept + { + static_assert(sizeof(v4_header) <= sizeof(self.buffer_), "buffer too small for v4 response"); + return boost::asio::buffer(self.buffer_, sizeof(v4_header)); + } + + void operator()(const boost::system::error_code error, const std::size_t bytes) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else if (bytes < self.buffer().size()) + self.done(socks::error::bad_write, std::move(self_)); + else + boost::asio::async_read(self.proxy_, get_buffer(self), completed{std::move(self_)}); + } + } + }; + + struct client::write + { + std::shared_ptr<client> self_; + + static boost::asio::const_buffers_1 get_buffer(client const& self) noexcept + { + return boost::asio::buffer(self.buffer_, self.buffer_size_); + } + + void operator()(const boost::system::error_code error) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else + boost::asio::async_write(self.proxy_, get_buffer(self), read{std::move(self_)}); + } + } + }; + + client::client(stream_type::socket&& proxy, socks::version ver) + : proxy_(std::move(proxy)), buffer_size_(0), buffer_(), ver_(ver) + {} + + client::~client() {} + + bool client::set_connect_command(const epee::net_utils::ipv4_network_address& address) + { + switch (socks_version()) + { + case version::v4: + case version::v4a: + case version::v4a_tor: + break; + default: + return false; + } + + static_assert(sizeof(v4_header) < sizeof(buffer_), "buffer size too small for request"); + static_assert(0 < sizeof(buffer_), "buffer size too small for null termination"); + + // version 4 + const v4_header temp{4, v4_connect_command, address.port(), boost::endian::big_to_native(address.ip())}; + std::memcpy(std::addressof(buffer_), std::addressof(temp), sizeof(temp)); + buffer_[sizeof(temp)] = 0; + buffer_size_ = sizeof(temp) + 1; + + return true; + } + + bool client::set_connect_command(const boost::string_ref domain, std::uint16_t port) + { + switch (socks_version()) + { + case version::v4a: + case version::v4a_tor: + break; + + default: + return false; + } + + const std::size_t buf_used = write_domain_header(buffer_, v4_connect_command, port, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::set_connect_command(const net::tor_address& address) + { + if (!address.is_unknown()) + return set_connect_command(address.host_str(), address.port()); + return false; + } + + bool client::set_resolve_command(boost::string_ref domain) + { + if (socks_version() != version::v4a_tor) + return false; + + const std::size_t buf_used = write_domain_header(buffer_, v4tor_resolve_command, 0, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + alias.proxy_.async_connect(proxy_address, write{std::move(self)}); + return true; + } + return false; + } + + bool client::send(std::shared_ptr<client> self) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + boost::asio::async_write(alias.proxy_, write::get_buffer(alias), read{std::move(self)}); + return true; + } + return false; + } +} // socks +} // net diff --git a/src/net/socks.h b/src/net/socks.h new file mode 100644 index 000000000..d29a51ccb --- /dev/null +++ b/src/net/socks.h @@ -0,0 +1,225 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/system/error_code.hpp> +#include <boost/type_traits/integral_constant.hpp> +#include <boost/utility/string_ref.hpp> +#include <memory> +#include <utility> + +#include "net/fwd.h" +#include "span.h" + +namespace epee +{ +namespace net_utils +{ + class ipv4_network_address; +} +} + +namespace net +{ +namespace socks +{ + //! Supported socks variants. + enum class version : std::uint8_t + { + v4 = 0, + v4a, + v4a_tor //!< Extensions defined in Tor codebase + }; + + //! Possible errors with socks communication. Defined in https://www.openssh.com/txt/socks4.protocol + enum class error : int + { + // 0 is reserved for success value + // 1-256 -> reserved for error values from socks server (+1 from wire value). + rejected = 92, + identd_connection, + identd_user, + // Specific to application + bad_read = 257, + bad_write, + unexpected_version + }; + + /* boost::system::error_code is extended for easier compatibility with + boost::asio errors. If std::error_code is needed (with expect<T> for + instance), then upgrade to boost 1.65+ or use conversion code in + develop branch at boost/system/detail/std_interoperability.hpp */ + + //! \return boost::system::error_category for net::socks namespace + const boost::system::error_category& error_category() noexcept; + + //! \return net::socks::error as a boost::system::error_code. + inline boost::system::error_code make_error_code(error value) noexcept + { + return boost::system::error_code{int(value), socks::error_category()}; + } + + //! Client support for socks connect and resolve commands. + class client + { + boost::asio::ip::tcp::socket proxy_; + std::uint16_t buffer_size_; + std::uint8_t buffer_[1024]; + socks::version ver_; + + /*! + Only invoked after `*send(...)` function completes or fails. + `bool(error) == false` indicates success; `self.get()` is always + `this` and allows implementations to skip + `std::enable_shared_from_this<T>` (ASIO callbacks need shared_ptr). + The design saves space and reduces cycles (everything uses moves, + so no atomic operations are ever necessary). + + \param error when processing last command (if any). + \param self `shared_ptr<client>` handle to `this`. + */ + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) = 0; + + public: + using stream_type = boost::asio::ip::tcp; + + // defined in cpp + struct write; + struct read; + struct completed; + + /*! + \param proxy ownership is passed into `this`. Does not have to be + in connected state. + \param ver socks version for the connection. + */ + explicit client(stream_type::socket&& proxy, socks::version ver); + + client(const client&) = delete; + virtual ~client(); + client& operator=(const client&) = delete; + + //! \return Ownership of socks client socket object. + stream_type::socket take_socket() + { + return stream_type::socket{std::move(proxy_)}; + } + + //! \return Socks version. + socks::version socks_version() const noexcept { return ver_; } + + //! \return Contents of internal buffer. + epee::span<const std::uint8_t> buffer() const noexcept + { + return {buffer_, buffer_size_}; + } + + //! \post `buffer.empty()`. + void clear_command() noexcept { buffer_size_ = 0; } + + //! Try to set `address` as remote connection request. + bool set_connect_command(const epee::net_utils::ipv4_network_address& address); + + //! Try to set `domain` + `port` as remote connection request. + bool set_connect_command(boost::string_ref domain, std::uint16_t port); + + //! Try to set `address` as remote Tor hidden service connection request. + bool set_connect_command(const net::tor_address& address); + + //! Try to set `domain` as remote DNS A record lookup request. + bool set_resolve_command(boost::string_ref domain); + + /*! + Asynchronously connect to `proxy_address` then issue command in + `buffer()`. The `done(...)` method will be invoked upon completion + with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + this function. + + \param self ownership of object is given to function. + \param proxy_address of the socks server. + \return False if `self->buffer().empty()` (no command set). + */ + static bool connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address); + + /*! + Assume existing connection to proxy server; asynchronously issue + command in `buffer()`. The `done(...)` method will be invoked + upon completion with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + the function. + + \param self ownership of object is given to function. + \return False if `self->buffer().empty()` (no command set). + */ + static bool send(std::shared_ptr<client> self); + }; + + template<typename Handler> + class connect_client : public client + { + Handler handler_; + + virtual void done(boost::system::error_code error, std::shared_ptr<client>) override + { + handler_(error, take_socket()); + } + + public: + explicit connect_client(stream_type::socket&& proxy, socks::version ver, Handler&& handler) + : client(std::move(proxy), ver), handler_(std::move(handler)) + {} + + virtual ~connect_client() override {} + }; + + template<typename Handler> + inline std::shared_ptr<client> + make_connect_client(client::stream_type::socket&& proxy, socks::version ver, Handler handler) + { + return std::make_shared<connect_client<Handler>>(std::move(proxy), ver, std::move(handler)); + } +} // socks +} // net + +namespace boost +{ +namespace system +{ + template<> + struct is_error_code_enum<net::socks::error> + : true_type + {}; +} // system +} // boost diff --git a/src/net/tor_address.cpp b/src/net/tor_address.cpp new file mode 100644 index 000000000..904a9a0fc --- /dev/null +++ b/src/net/tor_address.cpp @@ -0,0 +1,203 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "tor_address.h" + +#include <algorithm> +#include <boost/spirit/include/karma_generate.hpp> +#include <boost/spirit/include/karma_uint.hpp> +#include <cassert> +#include <cstring> +#include <limits> + +#include "net/error.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" +#include "string_tools.h" + +namespace net +{ + namespace + { + constexpr const char tld[] = u8".onion"; + constexpr const char unknown_host[] = "<unknown tor host>"; + + constexpr const unsigned v2_length = 16; + constexpr const unsigned v3_length = 56; + + constexpr const char base32_alphabet[] = + u8"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz234567"; + + expect<void> host_check(boost::string_ref host) noexcept + { + if (!host.ends_with(tld)) + return {net::error::expected_tld}; + + host.remove_suffix(sizeof(tld) - 1); + + //! \TODO v3 has checksum, base32 decoding is required to verify it + if (host.size() != v2_length && host.size() != v3_length) + return {net::error::invalid_tor_address}; + if (host.find_first_not_of(base32_alphabet) != boost::string_ref::npos) + return {net::error::invalid_tor_address}; + + return success(); + } + + struct tor_serialized + { + std::string host; + std::uint16_t port; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(host) + KV_SERIALIZE(port) + END_KV_SERIALIZE_MAP() + }; + } + + tor_address::tor_address(const boost::string_ref host, const std::uint16_t port) noexcept + : port_(port) + { + // this is a private constructor, throw if moved to public + assert(host.size() < sizeof(host_)); + + const std::size_t length = std::min(sizeof(host_) - 1, host.size()); + std::memcpy(host_, host.data(), length); + std::memset(host_ + length, 0, sizeof(host_) - length); + } + + const char* tor_address::unknown_str() noexcept + { + return unknown_host; + } + + tor_address::tor_address() noexcept + : port_(0) + { + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); + std::memset(host_ + sizeof(unknown_host), 0, sizeof(host_) - sizeof(unknown_host)); + } + + expect<tor_address> tor_address::make(const boost::string_ref address, const std::uint16_t default_port) + { + boost::string_ref host = address.substr(0, address.rfind(':')); + const boost::string_ref port = + address.substr(host.size() + (host.size() == address.size() ? 0 : 1)); + + MONERO_CHECK(host_check(host)); + + std::uint16_t porti = default_port; + if (!port.empty() && !epee::string_tools::get_xtype_from_string(porti, std::string{port})) + return {net::error::invalid_port}; + + static_assert(v2_length <= v3_length, "bad internal host size"); + static_assert(v3_length + sizeof(tld) == sizeof(tor_address::host_), "bad internal host size"); + return tor_address{host, porti}; + } + + bool tor_address::_load(epee::serialization::portable_storage& src, epee::serialization::section* hparent) + { + tor_serialized in{}; + if (in._load(src, hparent) && in.host.size() < sizeof(host_) && (in.host == unknown_host || !host_check(in.host).has_error())) + { + std::memcpy(host_, in.host.data(), in.host.size()); + std::memset(host_ + in.host.size(), 0, sizeof(host_) - in.host.size()); + port_ = in.port; + return true; + } + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); // include null terminator + port_ = 0; + return false; + } + + bool tor_address::store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const + { + const tor_serialized out{std::string{host_}, port_}; + return out.store(dest, hparent); + } + + tor_address::tor_address(const tor_address& rhs) noexcept + : port_(rhs.port_) + { + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + + tor_address& tor_address::operator=(const tor_address& rhs) noexcept + { + if (this != std::addressof(rhs)) + { + port_ = rhs.port_; + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + return *this; + } + + bool tor_address::is_unknown() const noexcept + { + static_assert(1 <= sizeof(host_), "host size too small"); + return host_[0] == '<'; // character is not allowed otherwise + } + + bool tor_address::equal(const tor_address& rhs) const noexcept + { + return port_ == rhs.port_ && is_same_host(rhs); + } + + bool tor_address::less(const tor_address& rhs) const noexcept + { + return std::strcmp(host_str(), rhs.host_str()) < 0 || port() < rhs.port(); + } + + bool tor_address::is_same_host(const tor_address& rhs) const noexcept + { + //! \TODO v2 and v3 should be comparable - requires base32 + return std::strcmp(host_str(), rhs.host_str()) == 0; + } + + std::string tor_address::str() const + { + const std::size_t host_length = std::strlen(host_str()); + const std::size_t port_length = + port_ == 0 ? 0 : std::numeric_limits<std::uint16_t>::digits10 + 2; + + std::string out{}; + out.reserve(host_length + port_length); + out.assign(host_str(), host_length); + + if (port_ != 0) + { + out.push_back(':'); + namespace karma = boost::spirit::karma; + karma::generate(std::back_inserter(out), karma::ushort_, port()); + } + return out; + } +} diff --git a/src/net/tor_address.h b/src/net/tor_address.h new file mode 100644 index 000000000..22d8cc119 --- /dev/null +++ b/src/net/tor_address.h @@ -0,0 +1,140 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> +#include <string> + +#include "common/expect.h" +#include "net/enums.h" +#include "net/error.h" + +namespace epee +{ +namespace serialization +{ + class portable_storage; + struct section; +} +} + +namespace net +{ + //! Tor onion address; internal format not condensed/decoded. + class tor_address + { + std::uint16_t port_; + char host_[63]; // null-terminated + + //! Keep in private, `host.size()` has no runtime check + tor_address(boost::string_ref host, std::uint16_t port) noexcept; + + public: + //! \return Size of internal buffer for host. + static constexpr std::size_t buffer_size() noexcept { return sizeof(host_); } + + //! \return `<unknown tor host>`. + static const char* unknown_str() noexcept; + + //! An object with `port() == 0` and `host_str() == unknown_str()`. + tor_address() noexcept; + + //! \return A default constructed `tor_address` object. + static tor_address unknown() noexcept { return tor_address{}; } + + /*! + Parse `address` in onion v2 or v3 format with (i.e. x.onion:80) + with `default_port` being used iff port is not specified in + `address`. + */ + static expect<tor_address> make(boost::string_ref address, std::uint16_t default_port = 0); + + //! Load from epee p2p format, and \return false if not valid tor address + bool _load(epee::serialization::portable_storage& src, epee::serialization::section* hparent); + + //! Store in epee p2p format + bool store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const; + + // Moves and copies are currently identical + + tor_address(const tor_address& rhs) noexcept; + ~tor_address() = default; + tor_address& operator=(const tor_address& rhs) noexcept; + + //! \return True if default constructed or via `unknown()`. + bool is_unknown() const noexcept; + + bool equal(const tor_address& rhs) const noexcept; + bool less(const tor_address& rhs) const noexcept; + + //! \return True if onion addresses are identical. + bool is_same_host(const tor_address& rhs) const noexcept; + + //! \return `x.onion` or `x.onion:z` if `port() != 0`. + std::string str() const; + + //! \return Null-terminated `x.onion` value or `unknown_str()`. + const char* host_str() const noexcept { return host_; } + + //! \return Port value or `0` if unspecified. + std::uint16_t port() const noexcept { return port_; } + + static constexpr bool is_loopback() noexcept { return false; } + static constexpr bool is_local() noexcept { return false; } + + static constexpr epee::net_utils::address_type get_type_id() noexcept + { + return epee::net_utils::address_type::tor; + } + + static constexpr epee::net_utils::zone get_zone() noexcept + { + return epee::net_utils::zone::tor; + } + + //! \return `!is_unknown()`. + bool is_blockable() const noexcept { return !is_unknown(); } + }; + + inline bool operator==(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.equal(rhs); + } + + inline bool operator!=(const tor_address& lhs, const tor_address& rhs) noexcept + { + return !lhs.equal(rhs); + } + + inline bool operator<(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.less(rhs); + } +} // net diff --git a/src/p2p/CMakeLists.txt b/src/p2p/CMakeLists.txt index 9b924907e..9a1730b8a 100644 --- a/src/p2p/CMakeLists.txt +++ b/src/p2p/CMakeLists.txt @@ -40,6 +40,7 @@ target_link_libraries(p2p PUBLIC version cryptonote_core + net ${UPNP_LIBRARIES} ${Boost_CHRONO_LIBRARY} ${Boost_PROGRAM_OPTIONS_LIBRARY} diff --git a/src/p2p/net_node.cpp b/src/p2p/net_node.cpp index 7cad6e077..8639fdb3b 100644 --- a/src/p2p/net_node.cpp +++ b/src/p2p/net_node.cpp @@ -28,9 +28,79 @@ // // Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers +#include <boost/algorithm/string/find_iterator.hpp> +#include <boost/algorithm/string/finder.hpp> +#include <boost/chrono/duration.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/optional/optional.hpp> +#include <boost/thread/future.hpp> +#include <boost/utility/string_ref.hpp> +#include <chrono> +#include <utility> + #include "common/command_line.h" #include "cryptonote_core/cryptonote_core.h" +#include "cryptonote_protocol/cryptonote_protocol_defs.h" #include "net_node.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "p2p/p2p_protocol_defs.h" +#include "string_tools.h" + +namespace +{ + constexpr const boost::chrono::milliseconds future_poll_interval{500}; + constexpr const std::chrono::seconds tor_connect_timeout{P2P_DEFAULT_TOR_CONNECT_TIMEOUT}; + + std::int64_t get_max_connections(const boost::iterator_range<boost::string_ref::const_iterator> value) noexcept + { + // -1 is default, 0 is error + if (value.empty()) + return -1; + + std::uint32_t out = 0; + if (epee::string_tools::get_xtype_from_string(out, std::string{value.begin(), value.end()})) + return out; + return 0; + } + + template<typename T> + epee::net_utils::network_address get_address(const boost::string_ref value) + { + expect<T> address = T::make(value); + if (!address) + { + MERROR( + "Failed to parse " << epee::net_utils::zone_to_string(T::get_zone()) << " address \"" << value << "\": " << address.error().message() + ); + return {}; + } + return {std::move(*address)}; + } + + bool start_socks(std::shared_ptr<net::socks::client> client, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + CHECK_AND_ASSERT_MES(client != nullptr, false, "Unexpected null client"); + + bool set = false; + switch (remote.get_type_id()) + { + case net::tor_address::get_type_id(): + set = client->set_connect_command(remote.as<net::tor_address>()); + break; + default: + MERROR("Unsupported network address in socks_connect"); + return false; + } + + const bool sent = + set && net::socks::client::connect_and_send(std::move(client), proxy); + CHECK_AND_ASSERT_MES(sent, false, "Unexpected failure to init socks client"); + return true; + } +} namespace nodetool { @@ -55,6 +125,8 @@ namespace nodetool const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node = {"add-exclusive-node", "Specify list of peers to connect to only." " If this option is given the options add-priority-node and seed-node are ignored"}; const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node = {"seed-node", "Connect to a node to retrieve peer addresses, and disconnect"}; + const command_line::arg_descriptor<std::vector<std::string> > arg_proxy = {"proxy", "<network-type>,<socks-ip:port>[,max_connections] i.e. \"tor,127.0.0.1:9050,100\""}; + const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound = {"anonymous-inbound", "<hidden-service-address>,<[bind-ip:]port>[,max_connections] i.e. \"x.onion,127.0.0.1:18083,100\""}; const command_line::arg_descriptor<bool> arg_p2p_hide_my_port = {"hide-my-port", "Do not announce yourself as peerlist candidate", false, true}; const command_line::arg_descriptor<bool> arg_no_igd = {"no-igd", "Disable UPnP port mapping"}; @@ -67,4 +139,196 @@ namespace nodetool const command_line::arg_descriptor<int64_t> arg_limit_rate = {"limit-rate", "set limit-rate [kB/s]", -1}; const command_line::arg_descriptor<bool> arg_save_graph = {"save-graph", "Save data for dr monero", false}; + + boost::optional<std::vector<proxy>> get_proxies(boost::program_options::variables_map const& vm) + { + namespace ip = boost::asio::ip; + + std::vector<proxy> proxies{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_proxy); + proxies.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + proxies.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No network type for --" << arg_proxy.name); + const boost::string_ref zone{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No ipv4:port given for --" << arg_proxy.name); + const boost::string_ref proxy{next->begin(), next->size()}; + + ++next; + if (!next.eof()) + { + proxies.back().max_connections = get_max_connections(*next); + if (proxies.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + switch (epee::net_utils::zone_from_string(zone)) + { + case epee::net_utils::zone::tor: + proxies.back().zone = epee::net_utils::zone::tor; + break; + default: + MERROR("Invalid network for --" << arg_proxy.name); + return boost::none; + } + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{proxy}) || port == 0) + { + MERROR("Invalid ipv4:port given for --" << arg_proxy.name); + return boost::none; + } + proxies.back().address = ip::tcp::endpoint{ip::address_v4{boost::endian::native_to_big(ip)}, port}; + } + + return proxies; + } + + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(boost::program_options::variables_map const& vm) + { + std::vector<anonymous_inbound> inbounds{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_anonymous_inbound); + inbounds.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + inbounds.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No inbound address for --" << arg_anonymous_inbound.name); + const boost::string_ref address{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No local ipv4:port given for --" << arg_anonymous_inbound.name); + const boost::string_ref bind{next->begin(), next->size()}; + + const std::size_t colon = bind.find_first_of(':'); + CHECK_AND_ASSERT_MES(colon < bind.size(), boost::none, "No local port given for --" << arg_anonymous_inbound.name); + + ++next; + if (!next.eof()) + { + inbounds.back().max_connections = get_max_connections(*next); + if (inbounds.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + expect<epee::net_utils::network_address> our_address = net::get_network_address(address, 0); + switch (our_address ? our_address->get_type_id() : epee::net_utils::address_type::invalid) + { + case net::tor_address::get_type_id(): + inbounds.back().our_address = std::move(*our_address); + inbounds.back().default_remote = net::tor_address::unknown(); + break; + default: + MERROR("Invalid inbound address (" << address << ") for --" << arg_anonymous_inbound.name << ": " << (our_address ? "invalid type" : our_address.error().message())); + return boost::none; + } + + // get_address returns default constructed address on error + if (inbounds.back().our_address == epee::net_utils::network_address{}) + return boost::none; + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{bind})) + { + MERROR("Invalid ipv4:port given for --" << arg_anonymous_inbound.name); + return boost::none; + } + inbounds.back().local_ip = std::string{bind.substr(0, colon)}; + inbounds.back().local_port = std::string{bind.substr(colon + 1)}; + } + + return inbounds; + } + + bool is_filtered_command(const epee::net_utils::network_address& address, int command) + { + switch (command) + { + case nodetool::COMMAND_HANDSHAKE_T<cryptonote::CORE_SYNC_DATA>::ID: + case nodetool::COMMAND_TIMED_SYNC_T<cryptonote::CORE_SYNC_DATA>::ID: + case cryptonote::NOTIFY_NEW_TRANSACTIONS::ID: + return false; + default: + break; + } + + if (address.get_zone() == epee::net_utils::zone::public_) + return false; + + MWARNING("Filtered command (#" << command << ") to/from " << address.str()); + return true; + } + + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + using socket_type = net::socks::client::stream_type::socket; + using client_result = std::pair<boost::system::error_code, socket_type>; + + struct notify + { + boost::promise<client_result> socks_promise; + + void operator()(boost::system::error_code error, socket_type&& sock) + { + socks_promise.set_value(std::make_pair(error, std::move(sock))); + } + }; + + boost::unique_future<client_result> socks_result{}; + { + boost::promise<client_result> socks_promise{}; + socks_result = socks_promise.get_future(); + + auto client = net::socks::make_connect_client( + boost::asio::ip::tcp::socket{service}, net::socks::version::v4a, notify{std::move(socks_promise)} + ); + if (!start_socks(std::move(client), proxy, remote)) + return boost::none; + } + + const auto start = std::chrono::steady_clock::now(); + while (socks_result.wait_for(future_poll_interval) == boost::future_status::timeout) + { + if (tor_connect_timeout < std::chrono::steady_clock::now() - start) + { + MERROR("Timeout on socks connect (" << proxy << " to " << remote.str() << ")"); + return boost::none; + } + + if (stop_signal) + return boost::none; + } + + try + { + auto result = socks_result.get(); + if (!result.first) + return {std::move(result.second)}; + + MERROR("Failed to make socks connection to " << remote.str() << " (via " << proxy << "): " << result.first.message()); + } + catch (boost::broken_promise const&) + {} + + return boost::none; + } } diff --git a/src/p2p/net_node.h b/src/p2p/net_node.h index 09c219a44..112f30fb6 100644 --- a/src/p2p/net_node.h +++ b/src/p2p/net_node.h @@ -30,12 +30,17 @@ #pragma once #include <array> +#include <atomic> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> #include <boost/thread.hpp> +#include <boost/optional/optional_fwd.hpp> #include <boost/program_options/options_description.hpp> #include <boost/program_options/variables_map.hpp> -#include <boost/serialization/version.hpp> #include <boost/uuid/uuid.hpp> -#include <boost/serialization/map.hpp> +#include <functional> +#include <utility> +#include <vector> #include "cryptonote_config.h" #include "warnings.h" @@ -47,6 +52,8 @@ #include "net_peerlist.h" #include "math_helper.h" #include "net_node_common.h" +#include "net/enums.h" +#include "net/fwd.h" #include "common/command_line.h" PUSH_WARNINGS @@ -54,6 +61,47 @@ DISABLE_VS_WARNINGS(4355) namespace nodetool { + struct proxy + { + proxy() + : max_connections(-1), + address(), + zone(epee::net_utils::zone::invalid) + {} + + std::int64_t max_connections; + boost::asio::ip::tcp::endpoint address; + epee::net_utils::zone zone; + }; + + struct anonymous_inbound + { + anonymous_inbound() + : max_connections(-1), + local_ip(), + local_port(), + our_address(), + default_remote() + {} + + std::int64_t max_connections; + std::string local_ip; + std::string local_port; + epee::net_utils::network_address our_address; + epee::net_utils::network_address default_remote; + }; + + boost::optional<std::vector<proxy>> get_proxies(const boost::program_options::variables_map& vm); + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(const boost::program_options::variables_map& vm); + + //! \return True if `commnd` is filtered (ignored/dropped) for `address` + bool is_filtered_command(epee::net_utils::network_address const& address, int command); + + // hides boost::future and chrono stuff from mondo template file + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote); + + template<class base_type> struct p2p_connection_context_t: base_type //t_payload_net_handler::connection_context //public net_utils::connection_context_base { @@ -78,53 +126,124 @@ namespace nodetool typedef COMMAND_HANDSHAKE_T<typename t_payload_net_handler::payload_type> COMMAND_HANDSHAKE; typedef COMMAND_TIMED_SYNC_T<typename t_payload_net_handler::payload_type> COMMAND_TIMED_SYNC; + typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context>> net_server; + + struct network_zone; + using connect_func = boost::optional<p2p_connection_context>(network_zone&, epee::net_utils::network_address const&); + + struct config + { + config() + : m_net_config(), + m_peer_id(crypto::rand<uint64_t>()), + m_support_flags(0) + {} + + network_config m_net_config; + uint64_t m_peer_id; + uint32_t m_support_flags; + }; + + struct network_zone + { + network_zone() + : m_connect(nullptr), + m_net_server(epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + network_zone(boost::asio::io_service& public_service) + : m_connect(nullptr), + m_net_server(public_service, epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + connect_func* m_connect; + net_server m_net_server; + std::string m_bind_ip; + std::string m_port; + epee::net_utils::network_address m_our_address; // in anonymity networks + peerlist_manager m_peerlist; + config m_config; + boost::asio::ip::tcp::endpoint m_proxy_address; + std::atomic<unsigned int> m_current_number_of_out_peers; + std::atomic<unsigned int> m_current_number_of_in_peers; + bool m_can_pingback; + + private: + void set_config_defaults() noexcept + { + // at this moment we have a hardcoded config + m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; + m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; + m_config.m_net_config.config_id = 0; + m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; + m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; + m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; + m_config.m_support_flags = 0; // only set in public zone + } + }; + public: typedef t_payload_net_handler payload_net_handler; node_server(t_payload_net_handler& payload_handler) - :m_payload_handler(payload_handler), - m_current_number_of_out_peers(0), - m_current_number_of_in_peers(0), - m_allow_local_ip(false), - m_hide_my_port(false), - m_no_igd(false), - m_offline(false), - m_save_graph(false), - is_closing(false), - m_net_server( epee::net_utils::e_connection_type_P2P ) // this is a P2P connection of the main p2p node server, because this is class node_server<> - {} - virtual ~node_server() + : m_payload_handler(payload_handler), + m_external_port(0), + m_allow_local_ip(false), + m_hide_my_port(false), + m_no_igd(false), + m_offline(false), + m_save_graph(false), + is_closing(false), + m_network_id() {} + virtual ~node_server(); static void init_options(boost::program_options::options_description& desc); bool run(); + network_zone& add_zone(epee::net_utils::zone zone); bool init(const boost::program_options::variables_map& vm); bool deinit(); bool send_stop_signal(); uint32_t get_this_peer_port(){return m_listening_port;} t_payload_net_handler& get_payload_object(); - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - a & m_peerlist; - if (ver == 0) - { - // from v1, we do not store the peer id anymore - peerid_type peer_id = AUTO_VAL_INIT (peer_id); - a & peer_id; - } - } // debug functions bool log_peerlist(); bool log_connections(); - virtual uint64_t get_connections_count(); - size_t get_outgoing_connections_count(); - size_t get_incoming_connections_count(); - peerlist_manager& get_peerlist_manager(){return m_peerlist;} - void delete_out_connections(size_t count); - void delete_in_connections(size_t count); + + // These functions only return information for the "public" zone + virtual uint64_t get_public_connections_count(); + size_t get_public_outgoing_connections_count(); + size_t get_public_white_peers_count(); + size_t get_public_gray_peers_count(); + void get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white); + size_t get_zone_count() const { return m_network_zones.size(); } + + void change_max_out_public_peers(size_t count); + void change_max_in_public_peers(size_t count); virtual bool block_host(const epee::net_utils::network_address &adress, time_t seconds = P2P_IP_BLOCKTIME); virtual bool unblock_host(const epee::net_utils::network_address &address); virtual std::map<std::string, time_t> get_blocked_hosts() { CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); return m_blocked_hosts; } @@ -150,6 +269,9 @@ namespace nodetool CHAIN_LEVIN_NOTIFY_MAP2(p2p_connection_context); //move levin_commands_handler interface notify(...) callbacks into nothing BEGIN_INVOKE_MAP2(node_server) + if (is_filtered_command(context.m_remote_address, command)) + return LEVIN_ERROR_CONNECTION_HANDLER_NOT_DEFINED; + HANDLE_INVOKE_T2(COMMAND_HANDSHAKE, &node_server::handle_handshake) HANDLE_INVOKE_T2(COMMAND_TIMED_SYNC, &node_server::handle_timed_sync) HANDLE_INVOKE_T2(COMMAND_PING, &node_server::handle_ping) @@ -178,7 +300,7 @@ namespace nodetool bool make_default_peer_id(); bool make_default_config(); bool store_config(); - bool check_trust(const proof_of_trust& tr); + bool check_trust(const proof_of_trust& tr, epee::net_utils::zone zone_type); //----------------- levin_commands_handler ------------------------------------------------------------- @@ -186,8 +308,7 @@ namespace nodetool virtual void on_connection_close(p2p_connection_context& context); virtual void callback(p2p_connection_context& context); //----------------- i_p2p_endpoint ------------------------------------------------------------- - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections); - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context); + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections); virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context); virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context); virtual bool drop_connection(const epee::net_utils::connection_context_base& context); @@ -204,7 +325,7 @@ namespace nodetool ); bool idle_worker(); bool handle_remote_peerlist(const std::vector<peerlist_entry>& peerlist, time_t local_time, const epee::net_utils::connection_context_base& context); - bool get_local_node_data(basic_node_data& node_data); + bool get_local_node_data(basic_node_data& node_data, const network_zone& zone); //bool get_local_handshake_data(handshake_data& hshd); bool merge_peerlist_with_local(const std::vector<peerlist_entry>& bs); @@ -216,7 +337,7 @@ namespace nodetool bool do_peer_timed_sync(const epee::net_utils::connection_context_base& context, peerid_type peer_id); bool make_new_connection_from_anchor_peerlist(const std::vector<anchor_peerlist_entry>& anchor_peerlist); - bool make_new_connection_from_peerlist(bool use_white_list); + bool make_new_connection_from_peerlist(network_zone& zone, bool use_white_list); bool try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist = false, uint64_t last_seen_stamp = 0, PeerType peer_type = white, uint64_t first_seen_stamp = 0); size_t get_random_index_with_fixed_probability(size_t max_index); bool is_peer_used(const peerlist_entry& peer); @@ -227,7 +348,7 @@ namespace nodetool template<class t_callback> bool try_ping(basic_node_data& node_data, p2p_connection_context& context, const t_callback &cb); bool try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f); - bool make_expected_connections_count(PeerType peer_type, size_t expected_connections); + bool make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections); void cache_connect_fail_info(const epee::net_utils::network_address& addr); bool is_addr_recently_failed(const epee::net_utils::network_address& addr); bool is_priority_node(const epee::net_utils::network_address& na); @@ -240,14 +361,8 @@ namespace nodetool template <class Container> bool parse_peers_and_add_to_container(const boost::program_options::variables_map& vm, const command_line::arg_descriptor<std::vector<std::string> > & arg, Container& container); - bool set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_out_peers() const { return m_config.m_net_config.max_out_connection_count; } - bool get_current_out_peers() const { return m_current_number_of_out_peers; } - - bool set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_in_peers() const { return m_config.m_net_config.max_in_connection_count; } - bool get_current_in_peers() const { return m_current_number_of_in_peers; } - + bool set_max_out_peers(network_zone& zone, int64_t max); + bool set_max_in_peers(network_zone& zone, int64_t max); bool set_tos_flag(const boost::program_options::variables_map& vm, int limit); bool set_rate_up_limit(const boost::program_options::variables_map& vm, int64_t limit); @@ -255,6 +370,11 @@ namespace nodetool bool set_rate_limit(const boost::program_options::variables_map& vm, int64_t limit); bool has_too_many_connections(const epee::net_utils::network_address &address); + uint64_t get_connections_count(); + size_t get_incoming_connections_count(); + size_t get_incoming_connections_count(network_zone&); + size_t get_outgoing_connections_count(); + size_t get_outgoing_connections_count(network_zone&); bool check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp); bool gray_peerlist_housekeeping(); @@ -272,25 +392,7 @@ namespace nodetool std::string print_connections_container(); - typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context> > net_server; - - struct config - { - network_config m_net_config; - uint64_t m_peer_id; - uint32_t m_support_flags; - - BEGIN_KV_SERIALIZE_MAP() - KV_SERIALIZE(m_net_config) - KV_SERIALIZE(m_peer_id) - KV_SERIALIZE(m_support_flags) - END_KV_SERIALIZE_MAP() - }; - public: - config m_config; // TODO was private, add getters? - std::atomic<unsigned int> m_current_number_of_out_peers; - std::atomic<unsigned int> m_current_number_of_in_peers; void set_save_graph(bool save_graph) { @@ -304,7 +406,6 @@ namespace nodetool bool m_first_connection_maker_call; uint32_t m_listening_port; uint32_t m_external_port; - uint32_t m_ip_address; bool m_allow_local_ip; bool m_hide_my_port; bool m_no_igd; @@ -316,7 +417,7 @@ namespace nodetool //connections_indexed_container m_connections; t_payload_net_handler& m_payload_handler; - peerlist_manager m_peerlist; + peerlist_storage m_peerlist_storage; epee::math_helper::once_a_time_seconds<P2P_DEFAULT_HANDSHAKE_INTERVAL> m_peer_handshake_idle_maker_interval; epee::math_helper::once_a_time_seconds<1> m_connections_maker_interval; @@ -324,8 +425,6 @@ namespace nodetool epee::math_helper::once_a_time_seconds<60> m_gray_peerlist_housekeeping_interval; epee::math_helper::once_a_time_seconds<3600, false> m_incoming_connections_interval; - std::string m_bind_ip; - std::string m_port; #ifdef ALLOW_DEBUG_COMMANDS uint64_t m_last_stat_request_time; #endif @@ -333,11 +432,22 @@ namespace nodetool std::vector<epee::net_utils::network_address> m_exclusive_peers; std::vector<epee::net_utils::network_address> m_seed_nodes; bool m_fallback_seed_nodes_added; - std::list<nodetool::peerlist_entry> m_command_line_peers; + std::vector<nodetool::peerlist_entry> m_command_line_peers; uint64_t m_peer_livetime; //keep connections to initiate some interactions - net_server m_net_server; - boost::uuids::uuid m_network_id; + + + static boost::optional<p2p_connection_context> public_connect(network_zone&, epee::net_utils::network_address const&); + static boost::optional<p2p_connection_context> socks_connect(network_zone&, epee::net_utils::network_address const&); + + + /* A `std::map` provides constant iterators and key/value pointers even with + inserts/erases to _other_ elements. This makes the configuration step easier + since references can safely be stored on the stack. Do not insert/erase + after configuration and before destruction, lock safety would need to be + added. `std::map::operator[]` WILL insert! */ + std::map<epee::net_utils::zone, network_zone> m_network_zones; + std::map<epee::net_utils::network_address, time_t> m_conn_fails_cache; epee::critical_section m_conn_fails_cache_lock; @@ -351,6 +461,7 @@ namespace nodetool boost::mutex m_used_stripe_peers_mutex; std::array<std::list<epee::net_utils::network_address>, 1 << CRYPTONOTE_PRUNING_LOG_STRIPES> m_used_stripe_peers; + boost::uuids::uuid m_network_id; cryptonote::network_type m_nettype; }; @@ -364,6 +475,8 @@ namespace nodetool extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_priority_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_proxy; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound; extern const command_line::arg_descriptor<bool> arg_p2p_hide_my_port; extern const command_line::arg_descriptor<bool> arg_no_igd; @@ -380,3 +493,4 @@ namespace nodetool } POP_WARNINGS + diff --git a/src/p2p/net_node.inl b/src/p2p/net_node.inl index c594984d4..471fdda0d 100644 --- a/src/p2p/net_node.inl +++ b/src/p2p/net_node.inl @@ -31,25 +31,34 @@ // IP blocking adapted from Boolberry #include <algorithm> +#include <boost/bind.hpp> #include <boost/date_time/posix_time/posix_time.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/optional/optional.hpp> #include <boost/thread/thread.hpp> #include <boost/uuid/uuid_io.hpp> -#include <boost/bind.hpp> #include <atomic> +#include <functional> +#include <limits> +#include <memory> +#include <tuple> +#include <vector> #include "version.h" #include "string_tools.h" #include "common/util.h" #include "common/dns_utils.h" #include "common/pruning.h" +#include "net/error.h" #include "net/net_helper.h" #include "math_helper.h" +#include "misc_log_ex.h" #include "p2p_protocol_defs.h" -#include "net_peerlist_boost_serialization.h" #include "net/local_ip.h" #include "crypto/crypto.h" #include "storages/levin_abstract_invoke2.h" #include "cryptonote_core/cryptonote_core.h" +#include "net/parse.h" #include <miniupnp/miniupnpc/miniupnpc.h> #include <miniupnp/miniupnpc/upnpcommands.h> @@ -64,6 +73,20 @@ namespace nodetool { + template<class t_payload_net_handler> + node_server<t_payload_net_handler>::~node_server() + { + // tcp server uses io_service in destructor, and every zone uses + // io_service from public zone. + for (auto current = m_network_zones.begin(); current != m_network_zones.end(); /* below */) + { + if (current->first != epee::net_utils::zone::public_) + current = m_network_zones.erase(current); + else + ++current; + } + } + //----------------------------------------------------------------------------------- inline bool append_net_address(std::vector<epee::net_utils::network_address> & seed_nodes, std::string const & addr, uint16_t default_port); //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -77,6 +100,8 @@ namespace nodetool command_line::add_arg(desc, arg_p2p_add_priority_node); command_line::add_arg(desc, arg_p2p_add_exclusive_node); command_line::add_arg(desc, arg_p2p_seed_node); + command_line::add_arg(desc, arg_proxy); + command_line::add_arg(desc, arg_anonymous_inbound); command_line::add_arg(desc, arg_p2p_hide_my_port); command_line::add_arg(desc, arg_no_igd); command_line::add_arg(desc, arg_out_peers); @@ -91,84 +116,14 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init_config() { - // TRY_ENTRY(); - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ifstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - // first try reading in portable mode - boost::archive::portable_binary_iarchive a(p2p_data); - a >> *this; - } - catch (...) - { - // if failed, try reading in unportable mode - boost::filesystem::copy_file(state_file_path, state_file_path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); - p2p_data.close(); - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - boost::archive::binary_iarchive a(p2p_data); - a >> *this; - } - catch (const std::exception &e) - { - MWARNING("Failed to load p2p config file, falling back to default config"); - m_peerlist = peerlist_manager(); // it was probably half clobbered by the failed load - make_default_config(); - } - } - else - { - make_default_config(); - } - } - }else - { - make_default_config(); - } - -#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED - std::list<peerlist_entry> plw; - while (m_peerlist.get_white_peers_count()) - { - plw.push_back(peerlist_entry()); - m_peerlist.get_white_peer_by_index(plw.back(), 0); - m_peerlist.remove_from_peer_white(plw.back()); - } - for (auto &e:plw) - m_peerlist.append_with_peer_white(e); - - std::list<peerlist_entry> plg; - while (m_peerlist.get_gray_peers_count()) - { - plg.push_back(peerlist_entry()); - m_peerlist.get_gray_peer_by_index(plg.back(), 0); - m_peerlist.remove_from_peer_gray(plg.back()); - } - for (auto &e:plg) - m_peerlist.append_with_peer_gray(e); -#endif - - // always recreate a new peer id - make_default_peer_id(); - - //at this moment we have hardcoded config - m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; - m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; //20 MB limit - m_config.m_net_config.config_id = 0; // initial config - m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; - m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; - m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; - m_config.m_support_flags = P2P_SUPPORT_FLAGS; + auto storage = peerlist_storage::open(m_config_folder + "/" + P2P_NET_DATA_FILENAME); + if (storage) + m_peerlist_storage = std::move(*storage); + m_network_zones[epee::net_utils::zone::public_].m_config.m_support_flags = P2P_SUPPORT_FLAGS; m_first_connection_maker_call = true; + CATCH_ENTRY_L0("node_server::init_config", false); return true; } @@ -176,17 +131,26 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::for_each_connection(std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + } } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::for_connection(const boost::uuids::uuid &connection_id, std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - return m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + const bool result = zone.second.m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + if (result) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -206,36 +170,33 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_peer_id() - { - m_config.m_peer_id = crypto::rand<uint64_t>(); - return true; - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_config() - { - return make_default_peer_id(); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::block_host(const epee::net_utils::network_address &addr, time_t seconds) { + if(!addr.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); m_blocked_hosts[addr.host_str()] = time(nullptr) + seconds; - // drop any connection to that IP - std::list<boost::uuids::uuid> conns; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + // drop any connection to that address. This should only have to look into + // the zone related to the connection, but really make sure everything is + // swept ... + std::vector<boost::uuids::uuid> conns; + for(auto& zone : m_network_zones) { - if (cntxt.m_remote_address.is_same_host(addr)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - conns.push_back(cntxt.m_connection_id); - } - return true; - }); - for (const auto &c: conns) - m_net_server.get_config_object().close(c); + if (cntxt.m_remote_address.is_same_host(addr)) + { + conns.push_back(cntxt.m_connection_id); + } + return true; + }); + for (const auto &c: conns) + zone.second.m_net_server.get_config_object().close(c); + + conns.clear(); + } MCLOG_CYAN(el::Level::Info, "global", "Host " << addr.host_str() << " blocked."); return true; @@ -256,6 +217,9 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::add_host_fail(const epee::net_utils::network_address &address) { + if(!address.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_host_fails_score_lock); uint64_t fails = ++m_host_fails_score[address.host_str()]; MDEBUG("Host " << address.host_str() << " fail score=" << fails); @@ -270,12 +234,6 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::parse_peer_from_string(epee::net_utils::network_address& pe, const std::string& node_addr, uint16_t default_port) - { - return epee::net_utils::create_network_address(pe, node_addr, default_port); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::handle_command_line( const boost::program_options::variables_map& vm ) @@ -284,8 +242,11 @@ namespace nodetool bool stagenet = command_line::get_arg(vm, cryptonote::arg_stagenet_on); m_nettype = testnet ? cryptonote::TESTNET : stagenet ? cryptonote::STAGENET : cryptonote::MAINNET; - m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); - m_port = command_line::get_arg(vm, arg_p2p_bind_port); + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + public_zone.m_connect = &public_connect; + public_zone.m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); + public_zone.m_port = command_line::get_arg(vm, arg_p2p_bind_port); + public_zone.m_can_pingback = true; m_external_port = command_line::get_arg(vm, arg_p2p_external_port); m_allow_local_ip = command_line::get_arg(vm, arg_p2p_allow_local_ip); m_no_igd = command_line::get_arg(vm, arg_no_igd); @@ -299,14 +260,20 @@ namespace nodetool nodetool::peerlist_entry pe = AUTO_VAL_INIT(pe); pe.id = crypto::rand<uint64_t>(); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(pe.adr, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - m_command_line_peers.push_back(pe); + add_zone(adr->get_zone()); + pe.adr = std::move(*adr); + m_command_line_peers.push_back(std::move(pe)); continue; } + CHECK_AND_ASSERT_MES( + adr == net::error::unsupported_address, false, "Bad address (\"" << pr_str << "\"): " << adr.error().message() + ); + std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -343,10 +310,13 @@ namespace nodetool if(command_line::has_arg(vm, arg_p2p_hide_my_port)) m_hide_my_port = true; - if ( !set_max_out_peers(vm, command_line::get_arg(vm, arg_out_peers) ) ) + if ( !set_max_out_peers(public_zone, command_line::get_arg(vm, arg_out_peers) ) ) return false; + else + m_payload_handler.set_max_out_peers(public_zone.m_config.m_net_config.max_out_connection_count); + - if ( !set_max_in_peers(vm, command_line::get_arg(vm, arg_in_peers) ) ) + if ( !set_max_in_peers(public_zone, command_line::get_arg(vm, arg_in_peers) ) ) return false; if ( !set_tos_flag(vm, command_line::get_arg(vm, arg_tos_flag) ) ) @@ -361,6 +331,58 @@ namespace nodetool if ( !set_rate_limit(vm, command_line::get_arg(vm, arg_limit_rate) ) ) return false; + + auto proxies = get_proxies(vm); + if (!proxies) + return false; + + for (auto& proxy : *proxies) + { + network_zone& zone = add_zone(proxy.zone); + if (zone.m_connect != nullptr) + { + MERROR("Listed --" << arg_proxy.name << " twice with " << epee::net_utils::zone_to_string(proxy.zone)); + return false; + } + zone.m_connect = &socks_connect; + zone.m_proxy_address = std::move(proxy.address); + + if (!set_max_out_peers(zone, proxy.max_connections)) + return false; + } + + for (const auto& zone : m_network_zones) + { + if (zone.second.m_connect == nullptr) + { + MERROR("Set outgoing peer for " << epee::net_utils::zone_to_string(zone.first) << " but did not set --" << arg_proxy.name); + return false; + } + } + + auto inbounds = get_anonymous_inbounds(vm); + if (!inbounds) + return false; + + for (auto& inbound : *inbounds) + { + network_zone& zone = add_zone(inbound.our_address.get_zone()); + + if (!zone.m_bind_ip.empty()) + { + MERROR("Listed --" << arg_anonymous_inbound.name << " twice with " << epee::net_utils::zone_to_string(inbound.our_address.get_zone()) << " network"); + return false; + } + + zone.m_bind_ip = std::move(inbound.local_ip); + zone.m_port = std::move(inbound.local_port); + zone.m_net_server.set_default_remote(std::move(inbound.default_remote)); + zone.m_our_address = std::move(inbound.our_address); + + if (!set_max_in_peers(zone, inbound.max_connections)) + return false; + } + return true; } //----------------------------------------------------------------------------------- @@ -442,7 +464,17 @@ namespace nodetool } return full_addrs; } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + typename node_server<t_payload_net_handler>::network_zone& node_server<t_payload_net_handler>::add_zone(const epee::net_utils::zone zone) + { + const auto zone_ = m_network_zones.lower_bound(zone); + if (zone_ != m_network_zones.end() && zone_->first == zone) + return zone_->second; + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + return m_network_zones.emplace_hint(zone_, std::piecewise_construct, std::make_tuple(zone), std::tie(public_zone.m_net_server.get_io_service()))->second; + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init(const boost::program_options::variables_map& vm) @@ -559,45 +591,81 @@ namespace nodetool MDEBUG("Number of seed nodes: " << m_seed_nodes.size()); m_config_folder = command_line::get_arg(vm, cryptonote::arg_data_dir); + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); - if ((m_nettype == cryptonote::MAINNET && m_port != std::to_string(::config::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::TESTNET && m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::STAGENET && m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { - m_config_folder = m_config_folder + "/" + m_port; + if ((m_nettype == cryptonote::MAINNET && public_zone.m_port != std::to_string(::config::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::TESTNET && public_zone.m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::STAGENET && public_zone.m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { + m_config_folder = m_config_folder + "/" + public_zone.m_port; } res = init_config(); CHECK_AND_ASSERT_MES(res, false, "Failed to init config."); - res = m_peerlist.init(m_allow_local_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + for (auto& zone : m_network_zones) + { + res = zone.second.m_peerlist.init(m_peerlist_storage.take_zone(zone.first), m_allow_local_ip); + CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + } + for(const auto& p: m_command_line_peers) + m_network_zones.at(p.adr.get_zone()).m_peerlist.append_with_peer_white(p); - for(auto& p: m_command_line_peers) - m_peerlist.append_with_peer_white(p); +// all peers are now setup +#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED + for (auto& zone : m_network_zones) + { + std::list<peerlist_entry> plw; + while (zone.second.m_peerlist.get_white_peers_count()) + { + plw.push_back(peerlist_entry()); + zone.second.m_peerlist.get_white_peer_by_index(plw.back(), 0); + zone.second.m_peerlist.remove_from_peer_white(plw.back()); + } + for (auto &e:plw) + zone.second.m_peerlist.append_with_peer_white(e); + + std::list<peerlist_entry> plg; + while (zone.second.m_peerlist.get_gray_peers_count()) + { + plg.push_back(peerlist_entry()); + zone.second.m_peerlist.get_gray_peer_by_index(plg.back(), 0); + zone.second.m_peerlist.remove_from_peer_gray(plg.back()); + } + for (auto &e:plg) + zone.second.m_peerlist.append_with_peer_gray(e); + } +#endif //only in case if we really sure that we have external visible ip m_have_address = true; - m_ip_address = 0; m_last_stat_request_time = 0; //configure self - m_net_server.set_threads_prefix("P2P"); - m_net_server.get_config_object().set_handler(this); - m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; - m_net_server.set_connection_filter(this); + + public_zone.m_net_server.set_threads_prefix("P2P"); // all zones use these threads/asio::io_service // from here onwards, it's online stuff if (m_offline) return res; //try to bind - MINFO("Binding on " << m_bind_ip << ":" << m_port); - res = m_net_server.init_server(m_port, m_bind_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().set_handler(this); + zone.second.m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; + + if (!zone.second.m_bind_ip.empty()) + { + zone.second.m_net_server.set_connection_filter(this); + MINFO("Binding on " << zone.second.m_bind_ip << ":" << zone.second.m_port); + res = zone.second.m_net_server.init_server(zone.second.m_port, zone.second.m_bind_ip); + CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + } + } - m_listening_port = m_net_server.get_binded_port(); - MLOG_GREEN(el::Level::Info, "Net service bound to " << m_bind_ip << ":" << m_listening_port); + m_listening_port = public_zone.m_net_server.get_binded_port(); + MLOG_GREEN(el::Level::Info, "Net service bound to " << public_zone.m_bind_ip << ":" << m_listening_port); if(m_external_port) MDEBUG("External port defined as " << m_external_port); @@ -621,44 +689,45 @@ namespace nodetool mPeersLoggerThread.reset(new boost::thread([&]() { _note("Thread monitor number of peers - start"); - while (!is_closing && !m_net_server.is_stop_signal_sent()) + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + while (!is_closing && !public_zone.m_net_server.is_stop_signal_sent()) { // main loop of thread //number_of_peers = m_net_server.get_config_object().get_connections_count(); - unsigned int number_of_in_peers = 0; - unsigned int number_of_out_peers = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - if (cntxt.m_is_income) - { - ++number_of_in_peers; - } - else + unsigned int number_of_in_peers = 0; + unsigned int number_of_out_peers = 0; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - ++number_of_out_peers; - } - return true; - }); // lambda - - m_current_number_of_in_peers = number_of_in_peers; - m_current_number_of_out_peers = number_of_out_peers; - + if (cntxt.m_is_income) + { + ++number_of_in_peers; + } + else + { + ++number_of_out_peers; + } + return true; + }); // lambda + zone.second.m_current_number_of_in_peers = number_of_in_peers; + zone.second.m_current_number_of_out_peers = number_of_out_peers; + } boost::this_thread::sleep_for(boost::chrono::seconds(1)); } // main loop of thread _note("Thread monitor number of peers - done"); })); // lambda + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + public_zone.m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); + public_zone.m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); + //here you can set worker threads count int thrds_count = 10; - - m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); - m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); - boost::thread::attributes attrs; attrs.set_stack_size(THREAD_STACK_SIZE); - //go to loop MINFO("Run net_service loop( " << thrds_count << " threads)..."); - if(!m_net_server.run_server(thrds_count, true, attrs)) + if(!public_zone.m_net_server.run_server(thrds_count, true, attrs)) { LOG_ERROR("Failed to run net tcp server!"); } @@ -666,23 +735,34 @@ namespace nodetool MINFO("net_service loop stopped."); return true; } - + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + uint64_t node_server<t_payload_net_handler>::get_public_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_net_server.get_config_object().get_connections_count(); + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> uint64_t node_server<t_payload_net_handler>::get_connections_count() { - return m_net_server.get_config_object().get_connections_count(); + std::uint64_t count = 0; + for (auto& zone : m_network_zones) + count += zone.second.m_net_server.get_config_object().get_connections_count(); + return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::deinit() { kill(); - m_peerlist.deinit(); if (!m_offline) { - m_net_server.deinit_server(); + for(auto& zone : m_network_zones) + zone.second.m_net_server.deinit_server(); // remove UPnP port mapping if(!m_no_igd) delete_upnp_port_mapping(m_listening_port); @@ -693,28 +773,25 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::store_config() { - TRY_ENTRY(); + if (!tools::create_directories_if_necessary(m_config_folder)) { - MWARNING("Failed to create data directory: " << m_config_folder); + MWARNING("Failed to create data directory \"" << m_config_folder); return false; } - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ofstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); - if(p2p_data.fail()) + peerlist_types active{}; + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(active); + + const std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; + if (!m_peerlist_storage.store(state_file_path, active)) { MWARNING("Failed to save config to file " << state_file_path); return false; - }; - - boost::archive::portable_binary_oarchive a(p2p_data); - a << *this; - return true; - CATCH_ENTRY_L0("blockchain_storage::save", false); - + } + CATCH_ENTRY_L0("node_server::store", false); return true; } //----------------------------------------------------------------------------------- @@ -722,36 +799,38 @@ namespace nodetool bool node_server<t_payload_net_handler>::send_stop_signal() { MDEBUG("[node] sending stop signal"); - m_net_server.send_stop_signal(); + for (auto& zone : m_network_zones) + zone.second.m_net_server.send_stop_signal(); MDEBUG("[node] Stop signal sent"); - std::list<boost::uuids::uuid> connection_ids; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - connection_ids.push_back(cntxt.m_connection_id); - return true; - }); - for (const auto &connection_id: connection_ids) - m_net_server.get_config_object().close(connection_id); - + for (auto& zone : m_network_zones) + { + std::list<boost::uuids::uuid> connection_ids; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { + connection_ids.push_back(cntxt.m_connection_id); + return true; + }); + for (const auto &connection_id: connection_ids) + zone.second.m_net_server.get_config_object().close(connection_id); + } m_payload_handler.stop(); - return true; } //----------------------------------------------------------------------------------- - - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::do_handshake_with_peer(peerid_type& pi, p2p_connection_context& context_, bool just_take_peerlist) { + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + typename COMMAND_HANDSHAKE::request arg; typename COMMAND_HANDSHAKE::response rsp; - get_local_node_data(arg.node_data); + get_local_node_data(arg.node_data, zone); m_payload_handler.get_payload_sync_data(arg.payload_data); epee::simple_event ev; std::atomic<bool> hsh_result(false); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, m_net_server.get_config_object(), + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, zone.m_net_server.get_config_object(), [this, &pi, &ev, &hsh_result, &just_take_peerlist, &context_](int code, const typename COMMAND_HANDSHAKE::response& rsp, p2p_connection_context& context) { epee::misc_utils::auto_scope_leave_caller scope_exit_handler = epee::misc_utils::create_scope_leave_handler([&](){ev.raise();}); @@ -785,13 +864,17 @@ namespace nodetool } pi = context.peer_id = rsp.node_data.peer_id; - m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); - if(rsp.node_data.peer_id == m_config.m_peer_id) + // move + for (auto const& zone : m_network_zones) { - LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); - hsh_result = false; - return; + if(rsp.node_data.peer_id == zone.second.m_config.m_peer_id) + { + LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); + hsh_result = false; + return; + } } LOG_INFO_CC(context, "New connection handshaked, pruning seed " << epee::string_tools::to_string_hex(context.m_pruning_seed)); LOG_DEBUG_CC(context, " COMMAND_HANDSHAKE INVOKED OK"); @@ -810,7 +893,7 @@ namespace nodetool if(!hsh_result) { LOG_WARNING_CC(context_, "COMMAND_HANDSHAKE Failed"); - m_net_server.get_config_object().close(context_.m_connection_id); + m_network_zones.at(context_.m_remote_address.get_zone()).m_net_server.get_config_object().close(context_.m_connection_id); } else { @@ -829,7 +912,8 @@ namespace nodetool typename COMMAND_TIMED_SYNC::request arg = AUTO_VAL_INIT(arg); m_payload_handler.get_payload_sync_data(arg.payload_data); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, zone.m_net_server.get_config_object(), [this](int code, const typename COMMAND_TIMED_SYNC::response& rsp, p2p_connection_context& context) { context.m_in_timedsync = false; @@ -842,11 +926,11 @@ namespace nodetool if(!handle_remote_peerlist(rsp.local_peerlist_new, rsp.local_time, context)) { LOG_WARNING_CC(context, "COMMAND_TIMED_SYNC: failed to handle_remote_peerlist(...), closing connection."); - m_net_server.get_config_object().close(context.m_connection_id ); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id ); add_host_fail(context.m_remote_address); } if(!context.m_is_income) - m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); m_payload_handler.process_payload_sync_data(rsp.payload_data, context, false); }); @@ -874,53 +958,61 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const peerlist_entry& peer) { - - if(m_config.m_peer_id == peer.id) - return true;//dont make connections to ourself + for(const auto& zone : m_network_zones) + if(zone.second.m_config.m_peer_id == peer.id) + return true;//dont make connections to ourself bool used = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - used = true; - return false;//stop enumerating - } - return true; - }); + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); - return used; + if(used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const anchor_peerlist_entry& peer) { - if(m_config.m_peer_id == peer.id) { - return true;//dont make connections to ourself - } - - bool used = false; - - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) - { - used = true; - - return false;//stop enumerating + for(auto& zone : m_network_zones) { + if(zone.second.m_config.m_peer_id == peer.id) { + return true;//dont make connections to ourself } - - return true; - }); - - return used; + bool used = false; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); + if (used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_addr_connected(const epee::net_utils::network_address& peer) { + const auto zone = m_network_zones.find(peer.get_zone()); + if (zone == m_network_zones.end()) + return false; + bool connected = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone->second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if(!cntxt.m_is_income && peer == cntxt.m_remote_address) { @@ -945,47 +1037,44 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist, uint64_t last_seen_stamp, PeerType peer_type, uint64_t first_seen_stamp) { - if (m_current_number_of_out_peers == m_config.m_net_config.max_out_connection_count) // out peers limit + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) // outgoing connections in zone not possible + return false; + + if (zone.m_current_number_of_out_peers == zone.m_config.m_net_config.max_out_connection_count) // out peers limit { return false; } - else if (m_current_number_of_out_peers > m_config.m_net_config.max_out_connection_count) + else if (zone.m_current_number_of_out_peers > zone.m_config.m_net_config.max_out_connection_count) { - m_net_server.get_config_object().del_out_connections(1); - m_current_number_of_out_peers --; // atomic variable, update time = 1s + zone.m_net_server.get_config_object().del_out_connections(1); + --(zone.m_current_number_of_out_peers); // atomic variable, update time = 1s return false; } + + MDEBUG("Connecting to " << na.str() << "(peer_type=" << peer_type << ", last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = peer_type == anchor; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if(!res) + auto con = zone.m_connect(zone, na); + if(!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str() + LOG_PRINT_CC_PRIORITY_NODE(is_priority, bool(con), "Connect failed to " << na.str() /*<< ", try " << try_count*/); //m_peerlist.set_peer_unreachable(pe); return false; } + con->m_anchor = peer_type == anchor; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, just_take_peerlist); + bool res = do_handshake_with_peer(pi, *con, just_take_peerlist); if(!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str() /*<< ", try " << try_count*/); return false; @@ -993,8 +1082,8 @@ namespace nodetool if(just_take_peerlist) { - m_net_server.get_config_object().close(con.m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + zone.m_net_server.get_config_object().close(con->m_connection_id); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1004,8 +1093,8 @@ namespace nodetool time_t last_seen; time(&last_seen); pe_local.last_seen = static_cast<int64_t>(last_seen); - pe_local.pruning_seed = con.m_pruning_seed; - m_peerlist.append_with_peer_white(pe_local); + pe_local.pruning_seed = con->m_pruning_seed; + zone.m_peerlist.append_with_peer_white(pe_local); //update last seen and push it to peerlist manager anchor_peerlist_entry ape = AUTO_VAL_INIT(ape); @@ -1013,52 +1102,46 @@ namespace nodetool ape.id = pi; ape.first_seen = first_seen_stamp ? first_seen_stamp : time(nullptr); - m_peerlist.append_with_peer_anchor(ape); + zone.m_peerlist.append_with_peer_anchor(ape); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK."); return true; } template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp) { + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) + return false; + LOG_PRINT_L1("Connecting to " << na.str() << "(last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = false; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if (!res) { + auto con = zone.m_connect(zone, na); + if (!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, p2p_connection_context{}, "Connect failed to " << na.str()); return false; } + con->m_anchor = false; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, true); - + const bool res = do_handshake_with_peer(pi, *con, true); if (!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str()); return false; } - m_net_server.get_config_object().close(con.m_connection_id); + zone.m_net_server.get_config_object().close(con->m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1115,7 +1198,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(bool use_white_list) + bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(network_zone& zone, bool use_white_list) { size_t max_random_index = 0; @@ -1123,7 +1206,7 @@ namespace nodetool size_t try_count = 0; size_t rand_count = 0; - while(rand_count < (max_random_index+1)*3 && try_count < 10 && !m_net_server.is_stop_signal_sent()) + while(rand_count < (max_random_index+1)*3 && try_count < 10 && !zone.m_net_server.is_stop_signal_sent()) { ++rand_count; size_t random_index; @@ -1132,7 +1215,7 @@ namespace nodetool std::deque<size_t> filtered; const size_t limit = use_white_list ? 20 : std::numeric_limits<size_t>::max(); size_t idx = 0; - m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ + zone.m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ if (filtered.size() >= limit) return false; if (next_needed_pruning_stripe == 0 || pe.pruning_seed == 0) @@ -1159,7 +1242,7 @@ namespace nodetool for (size_t i = 0; i < filtered.size(); ++i) { peerlist_entry pe; - if (m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) + if (zone.m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) { MDEBUG("Reusing stripe " << next_needed_pruning_stripe << " peer " << pe.adr.str()); random_index = i; @@ -1173,7 +1256,7 @@ namespace nodetool CHECK_AND_ASSERT_MES(random_index < filtered.size(), false, "random_index < filtered.size() failed!!"); random_index = filtered[random_index]; - CHECK_AND_ASSERT_MES(random_index < (use_white_list ? m_peerlist.get_white_peers_count() : m_peerlist.get_gray_peers_count()), + CHECK_AND_ASSERT_MES(random_index < (use_white_list ? zone.m_peerlist.get_white_peers_count() : zone.m_peerlist.get_gray_peers_count()), false, "random_index < peers size failed!!"); if(tried_peers.count(random_index)) @@ -1181,7 +1264,7 @@ namespace nodetool tried_peers.insert(random_index); peerlist_entry pe = AUTO_VAL_INIT(pe); - bool r = use_white_list ? m_peerlist.get_white_peer_by_index(pe, random_index):m_peerlist.get_gray_peer_by_index(pe, random_index); + bool r = use_white_list ? zone.m_peerlist.get_white_peer_by_index(pe, random_index):zone.m_peerlist.get_gray_peer_by_index(pe, random_index); CHECK_AND_ASSERT_MES(r, false, "Failed to get random peer from peerlist(white:" << use_white_list << ")"); ++try_count; @@ -1224,9 +1307,10 @@ namespace nodetool size_t try_count = 0; size_t current_index = crypto::rand<size_t>()%m_seed_nodes.size(); + const net_server& server = m_network_zones.at(epee::net_utils::zone::public_).m_net_server; while(true) { - if(m_net_server.is_stop_signal_sent()) + if(server.is_stop_signal_sent()) return false; if(try_to_connect_and_handshake_with_new_peer(m_seed_nodes[current_index], true)) @@ -1265,13 +1349,17 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::connections_maker() { + using zone_type = epee::net_utils::zone; + if (m_offline) return true; if (!connect_to_peerlist(m_exclusive_peers)) return false; if (!m_exclusive_peers.empty()) return true; - size_t start_conn_count = get_outgoing_connections_count(); - if(!m_peerlist.get_white_peers_count() && m_seed_nodes.size()) + // Only have seeds in the public zone right now. + + size_t start_conn_count = get_public_outgoing_connections_count(); + if(!get_public_white_peers_count() && m_seed_nodes.size()) { if (!connect_to_seed()) return false; @@ -1279,38 +1367,41 @@ namespace nodetool if (!connect_to_peerlist(m_priority_peers)) return false; - size_t base_expected_white_connections = (m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; - - size_t conn_count = get_outgoing_connections_count(); - while(conn_count < m_config.m_net_config.max_out_connection_count) + for(auto& zone : m_network_zones) { - const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? m_config.m_net_config.max_out_connection_count : base_expected_white_connections; - if(conn_count < expected_white_connections) - { - //start from anchor list - while (get_outgoing_connections_count() < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT - && make_expected_connections_count(anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); - //then do white list - while (get_outgoing_connections_count() < expected_white_connections - && make_expected_connections_count(white, expected_white_connections)); - //then do grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - }else + size_t base_expected_white_connections = (zone.second.m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; + + size_t conn_count = get_outgoing_connections_count(zone.second); + while(conn_count < zone.second.m_config.m_net_config.max_out_connection_count) { - //start from grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - //and then do white list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(white, m_config.m_net_config.max_out_connection_count)); + const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? zone.second.m_config.m_net_config.max_out_connection_count : base_expected_white_connections; + if(conn_count < expected_white_connections) + { + //start from anchor list + while (get_outgoing_connections_count(zone.second) < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT + && make_expected_connections_count(zone.second, anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); + //then do white list + while (get_outgoing_connections_count(zone.second) < expected_white_connections + && make_expected_connections_count(zone.second, white, expected_white_connections)); + //then do grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + }else + { + //start from grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + //and then do white list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, white, zone.second.m_config.m_net_config.max_out_connection_count)); + } + if(zone.second.m_net_server.is_stop_signal_sent()) + return false; + conn_count = get_outgoing_connections_count(zone.second); } - if(m_net_server.is_stop_signal_sent()) - return false; - conn_count = get_outgoing_connections_count(); } - if (start_conn_count == get_outgoing_connections_count() && start_conn_count < m_config.m_net_config.max_out_connection_count) + if (start_conn_count == get_public_outgoing_connections_count() && start_conn_count < m_network_zones.at(zone_type::public_).m_config.m_net_config.max_out_connection_count) { MINFO("Failed to connect to any, trying seeds"); if (!connect_to_seed()) @@ -1321,7 +1412,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_expected_connections_count(PeerType peer_type, size_t expected_connections) + bool node_server<t_payload_net_handler>::make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections) { if (m_offline) return false; @@ -1329,14 +1420,14 @@ namespace nodetool std::vector<anchor_peerlist_entry> apl; if (peer_type == anchor) { - m_peerlist.get_and_empty_anchor_peerlist(apl); + zone.m_peerlist.get_and_empty_anchor_peerlist(apl); } - size_t conn_count = get_outgoing_connections_count(); + size_t conn_count = get_outgoing_connections_count(zone); //add new connections from white peers if(conn_count < expected_connections) { - if(m_net_server.is_stop_signal_sent()) + if(zone.m_net_server.is_stop_signal_sent()) return false; MDEBUG("Making expected connection, type " << peer_type << ", " << conn_count << "/" << expected_connections << " connections"); @@ -1345,47 +1436,104 @@ namespace nodetool return false; } - if (peer_type == white && !make_new_connection_from_peerlist(true)) { + if (peer_type == white && !make_new_connection_from_peerlist(zone, true)) { return false; } - if (peer_type == gray && !make_new_connection_from_peerlist(false)) { + if (peer_type == gray && !make_new_connection_from_peerlist(zone, false)) { return false; } } return true; } - //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + size_t node_server<t_payload_net_handler>::get_public_outgoing_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return get_outgoing_connections_count(public_zone->second); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(!cntxt.m_is_income) + if(cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(cntxt.m_is_income) + if(!cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + { + size_t count = 0; + for(auto& zone : m_network_zones) + count += get_outgoing_connections_count(zone.second); + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + { + size_t count = 0; + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.m_is_income) + ++count; + return true; + }); + } + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_white_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_white_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_gray_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_gray_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + void node_server<t_payload_net_handler>::get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white) + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + public_zone->second.m_peerlist.get_peerlist(gray, white); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::idle_worker() { m_peer_handshake_idle_maker_interval.do_call(boost::bind(&node_server<t_payload_net_handler>::peer_sync_idle_maker, this)); @@ -1401,9 +1549,11 @@ namespace nodetool { if (m_offline) return true; - if (get_incoming_connections_count() == 0) + + const auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end() && get_incoming_connections_count(public_zone->second) == 0) { - if (m_hide_my_port || m_config.m_net_config.max_in_connection_count == 0) + if (m_hide_my_port || public_zone->second.m_config.m_net_config.max_in_connection_count == 0) { MGINFO("Incoming connections disabled, enable them for full connectivity"); } @@ -1422,15 +1572,18 @@ namespace nodetool MDEBUG("STARTED PEERLIST IDLE HANDSHAKE"); typedef std::list<std::pair<epee::net_utils::connection_context_base, peerid_type> > local_connects_type; local_connects_type cncts; - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id && !cntxt.m_in_timedsync) + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) { - cntxt.m_in_timedsync = true; - cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections - } - return true; - }); + if(cntxt.peer_id && !cntxt.m_in_timedsync) + { + cntxt.m_in_timedsync = true; + cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections + } + return true; + }); + } std::for_each(cncts.begin(), cncts.end(), [&](const typename local_connects_type::value_type& vl){do_peer_timed_sync(vl.first, vl.second);}); @@ -1468,19 +1621,30 @@ namespace nodetool std::vector<peerlist_entry> peerlist_ = peerlist; if(!fix_time_delta(peerlist_, local_time, delta)) return false; + + const epee::net_utils::zone zone = context.m_remote_address.get_zone(); + for(const auto& peer : peerlist_) + { + if(peer.adr.get_zone() != zone) + { + MWARNING(context << " sent peerlist from another zone, dropping"); + return false; + } + } + LOG_DEBUG_CC(context, "REMOTE PEERLIST: TIME_DELTA: " << delta << ", remote peerlist size=" << peerlist_.size()); LOG_DEBUG_CC(context, "REMOTE PEERLIST: " << print_peerlist_to_string(peerlist_)); - return m_peerlist.merge_peerlist(peerlist_); + return m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.merge_peerlist(peerlist_); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data) + bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data, const network_zone& zone) { time_t local_time; time(&local_time); - node_data.local_time = local_time; - node_data.peer_id = m_config.m_peer_id; - if(!m_hide_my_port) + node_data.local_time = local_time; // \TODO This can be an identifying value across zones (public internet to tor/i2p) ... + node_data.peer_id = zone.m_config.m_peer_id; + if(!m_hide_my_port && zone.m_can_pingback) node_data.my_port = m_external_port ? m_external_port : m_listening_port; else node_data.my_port = 0; @@ -1490,7 +1654,7 @@ namespace nodetool //----------------------------------------------------------------------------------- #ifdef ALLOW_DEBUG_COMMANDS template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr) + bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr, const epee::net_utils::zone zone_type) { uint64_t local_time = time(NULL); uint64_t time_delata = local_time > tr.time ? local_time - tr.time: tr.time - local_time; @@ -1504,9 +1668,11 @@ namespace nodetool MWARNING("check_trust failed to check time conditions, last_stat_request_time=" << m_last_stat_request_time << ", proof_time=" << tr.time); return false; } - if(m_config.m_peer_id != tr.peer_id) + + const network_zone& zone = m_network_zones.at(zone_type); + if(zone.m_config.m_peer_id != tr.peer_id) { - MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << m_config.m_peer_id<< ")"); + MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << zone.m_config.m_peer_id<< ")"); return false; } crypto::public_key pk = AUTO_VAL_INIT(pk); @@ -1525,12 +1691,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_stat_info(int command, typename COMMAND_REQUEST_STAT_INFO::request& arg, typename COMMAND_REQUEST_STAT_INFO::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - rsp.connections_count = m_net_server.get_config_object().get_connections_count(); + rsp.connections_count = get_connections_count(); rsp.incoming_connections_count = rsp.connections_count - get_outgoing_connections_count(); rsp.version = MONERO_VERSION_FULL; rsp.os_version = tools::get_os_version_string(); @@ -1541,12 +1707,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_network_state(int command, COMMAND_REQUEST_NETWORK_STATE::request& arg, COMMAND_REQUEST_NETWORK_STATE::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { connection_entry ce; ce.adr = cntxt.m_remote_address; @@ -1556,8 +1722,9 @@ namespace nodetool return true; }); - m_peerlist.get_peerlist_full(rsp.local_peerlist_gray, rsp.local_peerlist_white); - rsp.my_id = m_config.m_peer_id; + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + zone.m_peerlist.get_peerlist(rsp.local_peerlist_gray, rsp.local_peerlist_white); + rsp.my_id = zone.m_config.m_peer_id; rsp.local_time = time(NULL); return 1; } @@ -1565,7 +1732,7 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_peer_id(int command, COMMAND_REQUEST_PEER_ID::request& arg, COMMAND_REQUEST_PEER_ID::response& rsp, p2p_connection_context& context) { - rsp.my_id = m_config.m_peer_id; + rsp.my_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } #endif @@ -1573,40 +1740,42 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_support_flags(int command, COMMAND_REQUEST_SUPPORT_FLAGS::request& arg, COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context) { - rsp.support_flags = m_config.m_support_flags; + rsp.support_flags = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_support_flags; return 1; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> void node_server<t_payload_net_handler>::request_callback(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().request_callback(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().request_callback(context.m_connection_id); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections) + bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { + std::sort(connections.begin(), connections.end()); + auto zone = m_network_zones.begin(); for(const auto& c_id: connections) { - m_net_server.get_config_object().notify(command, data_buff, c_id); + for (;;) + { + if (zone == m_network_zones.end()) + { + MWARNING("Unable to relay all messages, " << epee::net_utils::zone_to_string(c_id.first) << " not available"); + return false; + } + if (c_id.first <= zone->first) + break; + + ++zone; + } + if (zone->first == c_id.first) + zone->second.m_net_server.get_config_object().notify(command, data_buff, c_id.second); } return true; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) - { - std::list<boost::uuids::uuid> connections; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id && context.m_connection_id != cntxt.m_connection_id) - connections.push_back(cntxt.m_connection_id); - return true; - }); - return relay_notify_to_list(command, data_buff, connections); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> void node_server<t_payload_net_handler>::callback(p2p_connection_context& context) { m_payload_handler.on_callback(context); @@ -1615,21 +1784,29 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::drop_connection(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().close(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id); return true; } //----------------------------------------------------------------------------------- @@ -1639,18 +1816,21 @@ namespace nodetool if(!node_data.my_port) return false; - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), false, "Only IPv4 addresses are supported here"); const epee::net_utils::network_address na = context.m_remote_address; uint32_t actual_ip = na.as<const epee::net_utils::ipv4_network_address>().ip(); - if(!m_peerlist.is_host_allowed(context.m_remote_address)) + network_zone& zone = m_network_zones.at(na.get_zone()); + + if(!zone.m_peerlist.is_host_allowed(context.m_remote_address)) return false; + std::string ip = epee::string_tools::get_ip_string_from_int32(actual_ip); std::string port = epee::string_tools::num_to_string_fast(node_data.my_port); epee::net_utils::network_address address{epee::net_utils::ipv4_network_address(actual_ip, node_data.my_port)}; peerid_type pr = node_data.peer_id; - bool r = m_net_server.connect_async(ip, port, m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( + bool r = zone.m_net_server.connect_async(ip, port, zone.m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( const typename net_server::t_connection_context& ping_context, const boost::system::error_code& ec)->bool { @@ -1670,7 +1850,9 @@ namespace nodetool // GCC 5.1.0 gives error with second use of uint64_t (peerid_type) variable. peerid_type pr_ = pr; - bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(address.get_zone()); + + bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, zone.m_net_server.get_config_object(), [=](int code, const COMMAND_PING::response& rsp, p2p_connection_context& context) { if(code <= 0) @@ -1679,20 +1861,21 @@ namespace nodetool return; } + network_zone& zone = m_network_zones.at(address.get_zone()); if(rsp.status != PING_OK_RESPONSE_STATUS_TEXT || pr != rsp.peer_id) { LOG_WARNING_CC(ping_context, "back ping invoke wrong response \"" << rsp.status << "\" from" << address.str() << ", hsh_peer_id=" << pr_ << ", rsp.peer_id=" << rsp.peer_id); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return; } - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); cb(); }); if(!inv_call_res) { LOG_WARNING_CC(ping_context, "back ping invoke failed to " << address.str()); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return false; } return true; @@ -1707,13 +1890,16 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f) { + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + return false; + COMMAND_REQUEST_SUPPORT_FLAGS::request support_flags_request; bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_REQUEST_SUPPORT_FLAGS::response> ( context.m_connection_id, COMMAND_REQUEST_SUPPORT_FLAGS::ID, support_flags_request, - m_net_server.get_config_object(), + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object(), [=](int code, const typename COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context_) { if(code < 0) @@ -1742,8 +1928,24 @@ namespace nodetool //fill response rsp.local_time = time(NULL); - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + + const epee::net_utils::zone zone_type = context.m_remote_address.get_zone(); + network_zone& zone = m_network_zones.at(zone_type); + + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); m_payload_handler.get_payload_sync_data(rsp.payload_data); + + /* Tor/I2P nodes receiving connections via forwarding (from tor/i2p daemon) + do not know the address of the connecting peer. This is relayed to them, + iff the node has setup an inbound hidden service. The other peer will have + to use the random peer_id value to link the two. My initial thought is that + the inbound peer should leave the other side marked as `<unknown tor host>`, + etc., because someone could give faulty addresses over Tor/I2P to get the + real peer with that identity banned/blacklisted. */ + + if(!context.m_is_income && zone.m_our_address.get_zone() == zone_type) + rsp.local_peerlist_new.push_back(peerlist_entry{zone.m_our_address, zone.m_config.m_peer_id, std::time(nullptr)}); + LOG_DEBUG_CC(context, "COMMAND_TIMED_SYNC"); return 1; } @@ -1775,7 +1977,9 @@ namespace nodetool return 1; } - if (m_current_number_of_in_peers >= m_config.m_net_config.max_in_connection_count) // in peers limit + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + + if (zone.m_current_number_of_in_peers >= zone.m_config.m_net_config.max_in_connection_count) // in peers limit { LOG_WARNING_CC(context, "COMMAND_HANDSHAKE came, but already have max incoming connections, so dropping this one."); drop_connection(context); @@ -1800,14 +2004,14 @@ namespace nodetool context.peer_id = arg.node_data.peer_id; context.m_in_timedsync = false; - if(arg.node_data.peer_id != m_config.m_peer_id && arg.node_data.my_port) + if(arg.node_data.peer_id != zone.m_config.m_peer_id && arg.node_data.my_port && zone.m_can_pingback) { peerid_type peer_id_l = arg.node_data.peer_id; uint32_t port_l = arg.node_data.my_port; //try ping to be sure that we can add this peer to peer_list try_ping(arg.node_data, context, [peer_id_l, port_l, context, this]() { - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, void(), + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), void(), "Only IPv4 addresses are supported here"); //called only(!) if success pinged, update local peerlist peerlist_entry pe; @@ -1818,7 +2022,7 @@ namespace nodetool pe.last_seen = static_cast<int64_t>(last_seen); pe.id = peer_id_l; pe.pruning_seed = context.m_pruning_seed; - this->m_peerlist.append_with_peer_white(pe); + this->m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.append_with_peer_white(pe); LOG_DEBUG_CC(context, "PING SUCCESS " << context.m_remote_address.host_str() << ":" << port_l); }); } @@ -1829,8 +2033,8 @@ namespace nodetool }); //fill response - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); - get_local_node_data(rsp.node_data); + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + get_local_node_data(rsp.node_data, zone); m_payload_handler.get_payload_sync_data(rsp.payload_data); LOG_DEBUG_CC(context, "COMMAND_HANDSHAKE"); return 1; @@ -1841,7 +2045,7 @@ namespace nodetool { LOG_DEBUG_CC(context, "COMMAND_PING"); rsp.status = PING_OK_RESPONSE_STATUS_TEXT; - rsp.peer_id = m_config.m_peer_id; + rsp.peer_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } //----------------------------------------------------------------------------------- @@ -1850,7 +2054,8 @@ namespace nodetool { std::vector<peerlist_entry> pl_white; std::vector<peerlist_entry> pl_gray; - m_peerlist.get_peerlist_full(pl_gray, pl_white); + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(pl_gray, pl_white); MINFO(ENDL << "Peerlist white:" << ENDL << print_peerlist_to_string(pl_white) << ENDL << "Peerlist gray:" << ENDL << print_peerlist_to_string(pl_gray) ); return true; } @@ -1867,14 +2072,17 @@ namespace nodetool { std::stringstream ss; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - ss << cntxt.m_remote_address.str() - << " \t\tpeer_id " << cntxt.peer_id - << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") - << std::endl; - return true; - }); + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + ss << cntxt.m_remote_address.str() + << " \t\tpeer_id " << cntxt.peer_id + << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") + << std::endl; + return true; + }); + } std::string s = ss.str(); return s; } @@ -1888,11 +2096,12 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::on_connection_close(p2p_connection_context& context) { - if (!m_net_server.is_stop_signal_sent() && !context.m_is_income) { + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + if (!zone.m_net_server.is_stop_signal_sent() && !context.m_is_income) { epee::net_utils::network_address na = AUTO_VAL_INIT(na); na = context.m_remote_address; - m_peerlist.remove_from_peer_anchor(na); + zone.m_peerlist.remove_from_peer_anchor(na); } m_payload_handler.on_connection_close(context); @@ -1909,9 +2118,10 @@ namespace nodetool template<class t_payload_net_handler> template <class Container> bool node_server<t_payload_net_handler>::connect_to_peerlist(const Container& peers) { + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); for(const epee::net_utils::network_address& na: peers) { - if(m_net_server.is_stop_signal_sent()) + if(public_zone.m_net_server.is_stop_signal_sent()) return false; if(is_addr_connected(na)) @@ -1930,16 +2140,16 @@ namespace nodetool for(const std::string& pr_str: perrs) { - epee::net_utils::network_address na = AUTO_VAL_INIT(na); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(na, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - container.push_back(na); + add_zone(adr->get_zone()); + container.push_back(std::move(*adr)); continue; } std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -1951,32 +2161,47 @@ namespace nodetool } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_out_peers(network_zone& zone, int64_t max) { - if(max == -1) - max = P2P_DEFAULT_CONNECTIONS_COUNT; - m_config.m_net_config.max_out_connection_count = max; - m_payload_handler.set_max_out_peers(max); + if(max == -1) { + zone.m_config.m_net_config.max_out_connection_count = P2P_DEFAULT_CONNECTIONS_COUNT; + return true; + } + zone.m_config.m_net_config.max_out_connection_count = max; return true; } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_in_peers(network_zone& zone, int64_t max) { - m_config.m_net_config.max_in_connection_count = max; + zone.m_config.m_net_config.max_in_connection_count = max; return true; } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_out_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_out_public_peers(size_t count) { - m_net_server.get_config_object().del_out_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_out_connection_count; + public_zone->second.m_config.m_net_config.max_out_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_out_connections(current - count); + } } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_in_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_in_public_peers(size_t count) { - m_net_server.get_config_object().del_in_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_in_connection_count; + public_zone->second.m_config.m_net_config.max_in_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_in_connections(current - count); + } } template<class t_payload_net_handler> @@ -2049,10 +2274,13 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::has_too_many_connections(const epee::net_utils::network_address &address) { + if (address.get_zone() != epee::net_utils::zone::public_) + return false; // Unable to determine how many connections from host + const size_t max_connections = 1; size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if (cntxt.m_is_income && cntxt.m_remote_address.is_same_host(address)) { count++; @@ -2075,29 +2303,29 @@ namespace nodetool if (!m_exclusive_peers.empty()) return true; if (m_payload_handler.needs_new_sync_connections()) return true; - peerlist_entry pe = AUTO_VAL_INIT(pe); - - if (m_net_server.is_stop_signal_sent()) - return false; - - if (!m_peerlist.get_random_gray_peer(pe)) { - return true; - } - - bool success = check_connection_and_handshake_with_peer(pe.adr, pe.last_seen); + for (auto& zone : m_network_zones) + { + if (zone.second.m_net_server.is_stop_signal_sent()) + return false; - if (!success) { - m_peerlist.remove_from_peer_gray(pe); + if (zone.second.m_connect == nullptr) + continue; - LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + peerlist_entry pe{}; + if (!zone.second.m_peerlist.get_random_gray_peer(pe)) + continue; - return true; + if (!check_connection_and_handshake_with_peer(pe.adr, pe.last_seen)) + { + zone.second.m_peerlist.remove_from_peer_gray(pe); + LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } + else + { + zone.second.m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); + LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } } - - m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); - - LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); - return true; } @@ -2225,4 +2453,37 @@ namespace nodetool MINFO("No IGD was found."); } } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::socks_connect(network_zone& zone, const epee::net_utils::network_address& remote) + { + auto result = socks_connect_internal(zone.m_net_server.get_stop_signal(), zone.m_net_server.get_io_service(), zone.m_proxy_address, remote); + if (result) // if no error + { + p2p_connection_context context{}; + if (zone.m_net_server.add_connection(context, std::move(*result), remote)) + return {std::move(context)}; + } + return boost::none; + } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::public_connect(network_zone& zone, epee::net_utils::network_address const& na) + { + CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), boost::none, + "Only IPv4 addresses are supported here"); + const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); + + typename net_server::t_connection_context con{}; + const bool res = zone.m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), + epee::string_tools::num_to_string_fast(ipv4.port()), + zone.m_config.m_net_config.connection_timeout, + con); + + if (res) + return {std::move(con)}; + return boost::none; + } } diff --git a/src/p2p/net_node_common.h b/src/p2p/net_node_common.h index 075c18ef9..944bf48e4 100644 --- a/src/p2p/net_node_common.h +++ b/src/p2p/net_node_common.h @@ -31,6 +31,8 @@ #pragma once #include <boost/uuid/uuid.hpp> +#include <utility> +#include <vector> #include "net/net_utils_base.h" #include "p2p_protocol_defs.h" @@ -43,13 +45,13 @@ namespace nodetool template<class t_connection_context> struct i_p2p_endpoint { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections)=0; - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context)=0; + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections)=0; virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool drop_connection(const epee::net_utils::connection_context_base& context)=0; virtual void request_callback(const epee::net_utils::connection_context_base& context)=0; - virtual uint64_t get_connections_count()=0; + virtual uint64_t get_public_connections_count()=0; + virtual size_t get_zone_count() const=0; virtual void for_each_connection(std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool for_connection(const boost::uuids::uuid&, std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool block_host(const epee::net_utils::network_address &address, time_t seconds = 0)=0; @@ -64,11 +66,7 @@ namespace nodetool template<class t_connection_context> struct p2p_endpoint_stub: public i_p2p_endpoint<t_connection_context> { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections) - { - return false; - } - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { return false; } @@ -97,7 +95,12 @@ namespace nodetool return false; } - virtual uint64_t get_connections_count() + virtual size_t get_zone_count() const + { + return 0; + } + + virtual uint64_t get_public_connections_count() { return false; } diff --git a/src/p2p/net_peerlist.cpp b/src/p2p/net_peerlist.cpp new file mode 100644 index 000000000..ce5c67fe5 --- /dev/null +++ b/src/p2p/net_peerlist.cpp @@ -0,0 +1,295 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "net_peerlist.h" + +#include <algorithm> +#include <functional> +#include <fstream> +#include <iterator> + +#include <boost/archive/binary_iarchive.hpp> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/range/join.hpp> +#include <boost/serialization/version.hpp> + +#include "net_peerlist_boost_serialization.h" + + +namespace nodetool +{ + namespace + { + constexpr unsigned CURRENT_PEERLIST_STORAGE_ARCHIVE_VER = 6; + + struct by_zone + { + using zone = epee::net_utils::zone; + + template<typename T> + bool operator()(const T& left, const zone right) const + { + return left.adr.get_zone() < right; + } + + template<typename T> + bool operator()(const zone left, const T& right) const + { + return left < right.adr.get_zone(); + } + + template<typename T, typename U> + bool operator()(const T& left, const U& right) const + { + return left.adr.get_zone() < right.adr.get_zone(); + } + }; + + template<typename Elem, typename Archive> + std::vector<Elem> load_peers(Archive& a, unsigned ver) + { + // at v6, we drop existing peerlists, because annoying change + if (ver < 6) + return {}; + + uint64_t size = 0; + a & size; + + Elem ple{}; + + std::vector<Elem> elems{}; + elems.reserve(size); + while (size--) + { + a & ple; + elems.push_back(std::move(ple)); + } + + return elems; + } + + template<typename Archive, typename Range> + void save_peers(Archive& a, const Range& elems) + { + const uint64_t size = elems.size(); + a & size; + for (const auto& elem : elems) + a & elem; + } + + template<typename T> + std::vector<T> do_take_zone(std::vector<T>& src, epee::net_utils::zone zone) + { + const auto start = std::lower_bound(src.begin(), src.end(), zone, by_zone{}); + const auto end = std::upper_bound(start, src.end(), zone, by_zone{}); + + std::vector<T> out{}; + out.assign(std::make_move_iterator(start), std::make_move_iterator(end)); + src.erase(start, end); + return out; + } + + template<typename Container, typename T> + void add_peers(Container& dest, std::vector<T>&& src) + { + dest.insert(std::make_move_iterator(src.begin()), std::make_move_iterator(src.end())); + } + + template<typename Container, typename Range> + void copy_peers(Container& dest, const Range& src) + { + std::copy(src.begin(), src.end(), std::back_inserter(dest)); + } + } // anonymous + + struct peerlist_join + { + const peerlist_types& ours; + const peerlist_types& other; + }; + + template<typename Archive> + void serialize(Archive& a, peerlist_types& elem, unsigned ver) + { + elem.white = load_peers<peerlist_entry>(a, ver); + elem.gray = load_peers<peerlist_entry>(a, ver); + elem.anchor = load_peers<anchor_peerlist_entry>(a, ver); + + if (ver == 0) + { + // from v1, we do not store the peer id anymore + peerid_type peer_id{}; + a & peer_id; + } + } + + template<typename Archive> + void serialize(Archive& a, peerlist_join elem, unsigned ver) + { + save_peers(a, boost::range::join(elem.ours.white, elem.other.white)); + save_peers(a, boost::range::join(elem.ours.gray, elem.other.gray)); + save_peers(a, boost::range::join(elem.ours.anchor, elem.other.anchor)); + } + + boost::optional<peerlist_storage> peerlist_storage::open(std::istream& src, const bool new_format) + { + try + { + peerlist_storage out{}; + if (new_format) + { + boost::archive::portable_binary_iarchive a{src}; + a >> out.m_types; + } + else + { + boost::archive::binary_iarchive a{src}; + a >> out.m_types; + } + + if (src.good()) + { + std::sort(out.m_types.white.begin(), out.m_types.white.end(), by_zone{}); + std::sort(out.m_types.gray.begin(), out.m_types.gray.end(), by_zone{}); + std::sort(out.m_types.anchor.begin(), out.m_types.anchor.end(), by_zone{}); + return {std::move(out)}; + } + } + catch (const std::exception& e) + {} + + return boost::none; + } + + boost::optional<peerlist_storage> peerlist_storage::open(const std::string& path) + { + std::ifstream src_file{}; + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + boost::optional<peerlist_storage> out = open(src_file, true); + if (!out) + { + // if failed, try reading in unportable mode + boost::filesystem::copy_file(path, path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); + src_file.close(); + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + out = open(src_file, false); + if (!out) + { + // This is different from the `return boost::none` cases above. Those + // cases could fail due to bad file permissions, so a shutdown is + // likely more appropriate. + MWARNING("Failed to load p2p config file, falling back to default config"); + out.emplace(); + } + } + + return out; + } + + peerlist_storage::~peerlist_storage() noexcept + {} + + bool peerlist_storage::store(std::ostream& dest, const peerlist_types& other) const + { + try + { + boost::archive::portable_binary_oarchive a{dest}; + const peerlist_join pj{std::cref(m_types), std::cref(other)}; + a << pj; + return dest.good(); + } + catch (const boost::archive::archive_exception& e) + {} + + return false; + } + + bool peerlist_storage::store(const std::string& path, const peerlist_types& other) const + { + std::ofstream dest_file{}; + dest_file.open( path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); + if(dest_file.fail()) + return false; + + return store(dest_file, other); + } + + peerlist_types peerlist_storage::take_zone(epee::net_utils::zone zone) + { + peerlist_types out{}; + out.white = do_take_zone(m_types.white, zone); + out.gray = do_take_zone(m_types.gray, zone); + out.anchor = do_take_zone(m_types.anchor, zone); + return out; + } + + bool peerlist_manager::init(peerlist_types&& peers, bool allow_local_ip) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + + if (!m_peers_white.empty() || !m_peers_gray.empty() || !m_peers_anchor.empty()) + return false; + + add_peers(m_peers_white.get<by_addr>(), std::move(peers.white)); + add_peers(m_peers_gray.get<by_addr>(), std::move(peers.gray)); + add_peers(m_peers_anchor.get<by_addr>(), std::move(peers.anchor)); + m_allow_local_ip = allow_local_ip; + return true; + } + + void peerlist_manager::get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + copy_peers(pl_gray, m_peers_gray.get<by_addr>()); + copy_peers(pl_white, m_peers_white.get<by_addr>()); + } + + void peerlist_manager::get_peerlist(peerlist_types& peers) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + peers.white.reserve(peers.white.size() + m_peers_white.size()); + peers.gray.reserve(peers.gray.size() + m_peers_gray.size()); + peers.anchor.reserve(peers.anchor.size() + m_peers_anchor.size()); + + copy_peers(peers.white, m_peers_white.get<by_addr>()); + copy_peers(peers.gray, m_peers_gray.get<by_addr>()); + copy_peers(peers.anchor, m_peers_anchor.get<by_addr>()); + } +} + +BOOST_CLASS_VERSION(nodetool::peerlist_types, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); +BOOST_CLASS_VERSION(nodetool::peerlist_join, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); + diff --git a/src/p2p/net_peerlist.h b/src/p2p/net_peerlist.h index 685fdc193..46726d7d8 100644 --- a/src/p2p/net_peerlist.h +++ b/src/p2p/net_peerlist.h @@ -30,33 +30,67 @@ #pragma once +#include <iosfwd> #include <list> -#include <set> -#include <map> -#include <boost/archive/binary_iarchive.hpp> -#include <boost/archive/portable_binary_oarchive.hpp> -#include <boost/archive/portable_binary_iarchive.hpp> -#include <boost/serialization/version.hpp> +#include <string> +#include <vector> #include <boost/multi_index_container.hpp> #include <boost/multi_index/ordered_index.hpp> #include <boost/multi_index/identity.hpp> #include <boost/multi_index/member.hpp> +#include <boost/optional/optional.hpp> #include <boost/range/adaptor/reversed.hpp> -#include "syncobj.h" +#include "cryptonote_config.h" +#include "net/enums.h" #include "net/local_ip.h" #include "p2p_protocol_defs.h" -#include "cryptonote_config.h" -#include "net_peerlist_boost_serialization.h" - - -#define CURRENT_PEERLIST_STORAGE_ARCHIVE_VER 6 +#include "syncobj.h" namespace nodetool { + struct peerlist_types + { + std::vector<peerlist_entry> white; + std::vector<peerlist_entry> gray; + std::vector<anchor_peerlist_entry> anchor; + }; + + class peerlist_storage + { + public: + peerlist_storage() + : m_types{} + {} + + //! \return Peers stored in stream `src` in `new_format` (portable archive or older non-portable). + static boost::optional<peerlist_storage> open(std::istream& src, const bool new_format); + + //! \return Peers stored in file at `path` + static boost::optional<peerlist_storage> open(const std::string& path); + + peerlist_storage(peerlist_storage&&) = default; + peerlist_storage(const peerlist_storage&) = delete; + + ~peerlist_storage() noexcept; + peerlist_storage& operator=(peerlist_storage&&) = default; + peerlist_storage& operator=(const peerlist_storage&) = delete; + + //! Save peers from `this` and `other` in stream `dest`. + bool store(std::ostream& dest, const peerlist_types& other) const; + + //! Save peers from `this` and `other` in one file at `path`. + bool store(const std::string& path, const peerlist_types& other) const; + + //! \return Peers in `zone` and from remove from `this`. + peerlist_types take_zone(epee::net_utils::zone zone); + + private: + peerlist_types m_types; + }; /************************************************************************/ /* */ @@ -64,13 +98,13 @@ namespace nodetool class peerlist_manager { public: - bool init(bool allow_local_ip); - bool deinit(); + bool init(peerlist_types&& peers, bool allow_local_ip); size_t get_white_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_white.size();} size_t get_gray_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_gray.size();} bool merge_peerlist(const std::vector<peerlist_entry>& outer_bs); bool get_peerlist_head(std::vector<peerlist_entry>& bs_head, uint32_t depth = P2P_DEFAULT_PEERS_IN_HANDSHAKE); - bool get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(peerlist_types& peers); bool get_white_peer_by_index(peerlist_entry& p, size_t i); bool get_gray_peer_by_index(peerlist_entry& p, size_t i); template<typename F> bool foreach(bool white, const F &f); @@ -136,18 +170,6 @@ namespace nodetool > peers_indexed; typedef boost::multi_index_container< - peerlist_entry, - boost::multi_index::indexed_by< - // access by peerlist_entry::id< - boost::multi_index::ordered_unique<boost::multi_index::tag<by_id>, boost::multi_index::member<peerlist_entry,uint64_t,&peerlist_entry::id> >, - // access by peerlist_entry::net_adress - boost::multi_index::ordered_unique<boost::multi_index::tag<by_addr>, boost::multi_index::member<peerlist_entry,epee::net_utils::network_address,&peerlist_entry::adr> >, - // sort by peerlist_entry::last_seen< - boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<peerlist_entry,int64_t,&peerlist_entry::last_seen> > - > - > peers_indexed_old; - - typedef boost::multi_index_container< anchor_peerlist_entry, boost::multi_index::indexed_by< // access by anchor_peerlist_entry::net_adress @@ -156,56 +178,8 @@ namespace nodetool boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<anchor_peerlist_entry,int64_t,&anchor_peerlist_entry::first_seen> > > > anchor_peers_indexed; - public: - - template <class Archive, class List, class Element, class t_version_type> - void serialize_peers(Archive &a, List &list, Element ple, const t_version_type ver) - { - if (typename Archive::is_saving()) - { - uint64_t size = list.size(); - a & size; - for (auto p: list) - { - a & p; - } - } - else - { - uint64_t size; - a & size; - list.clear(); - while (size--) - { - a & ple; - list.insert(ple); - } - } - } - - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - // at v6, we drop existing peerlists, because annoying change - if (ver < 6) - return; - - CRITICAL_REGION_LOCAL(m_peerlist_lock); - -#if 0 - // trouble loading more than one peer, can't find why - a & m_peers_white; - a & m_peers_gray; - a & m_peers_anchor; -#else - serialize_peers(a, m_peers_white, peerlist_entry(), ver); - serialize_peers(a, m_peers_gray, peerlist_entry(), ver); - serialize_peers(a, m_peers_anchor, anchor_peerlist_entry(), ver); -#endif - } private: - bool peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi); void trim_white_peerlist(); void trim_gray_peerlist(); @@ -220,34 +194,6 @@ namespace nodetool anchor_peers_indexed m_peers_anchor; }; //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::init(bool allow_local_ip) - { - m_allow_local_ip = allow_local_ip; - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::deinit() - { - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi) - { - for(auto x: pio) - { - auto by_addr_it = pi.get<by_addr>().find(x.adr); - if(by_addr_it == pi.get<by_addr>().end()) - { - pi.insert(x); - } - } - - return true; - } - //-------------------------------------------------------------------------------------------------- inline void peerlist_manager::trim_gray_peerlist() { while(m_peers_gray.size() > P2P_LOCAL_GRAY_PEERLIST_LIMIT) @@ -337,27 +283,6 @@ namespace nodetool return true; } //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) - { - CRITICAL_REGION_LOCAL(m_peerlist_lock); - peers_indexed::index<by_time>::type& by_time_index_gr=m_peers_gray.get<by_time>(); - pl_gray.resize(pl_gray.size() + by_time_index_gr.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_gr)) - { - pl_gray.push_back(vl); - } - - peers_indexed::index<by_time>::type& by_time_index_wt=m_peers_white.get<by_time>(); - pl_white.resize(pl_white.size() + by_time_index_wt.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_wt)) - { - pl_white.push_back(vl); - } - - return true; - } - //-------------------------------------------------------------------------------------------------- template<typename F> inline bool peerlist_manager::foreach(bool white, const F &f) { @@ -559,4 +484,3 @@ namespace nodetool //-------------------------------------------------------------------------------------------------- } -BOOST_CLASS_VERSION(nodetool::peerlist_manager, CURRENT_PEERLIST_STORAGE_ARCHIVE_VER) diff --git a/src/p2p/net_peerlist_boost_serialization.h b/src/p2p/net_peerlist_boost_serialization.h index 2f4a7e661..d2e9efa3d 100644 --- a/src/p2p/net_peerlist_boost_serialization.h +++ b/src/p2p/net_peerlist_boost_serialization.h @@ -1,4 +1,4 @@ -// Copyright (c) 2014-2018, The Monero Project + // Copyright (c) 2014-2018, The Monero Project // // All rights reserved. // @@ -30,7 +30,11 @@ #pragma once +#include <cstring> + +#include "common/expect.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" #include "p2p/p2p_protocol_defs.h" #ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED @@ -42,24 +46,37 @@ namespace boost namespace serialization { template <class T, class Archive> - inline void do_serialize(Archive &a, epee::net_utils::network_address& na, T local) + inline void do_serialize(boost::mpl::false_, Archive &a, epee::net_utils::network_address& na) { - if (typename Archive::is_saving()) local = na.as<T>(); - a & local; - if (!typename Archive::is_saving()) na = local; + T addr{}; + a & addr; + na = std::move(addr); } + + template <class T, class Archive> + inline void do_serialize(boost::mpl::true_, Archive &a, const epee::net_utils::network_address& na) + { + a & na.as<T>(); + } + template <class Archive, class ver_type> inline void serialize(Archive &a, epee::net_utils::network_address& na, const ver_type ver) { + static constexpr const typename Archive::is_saving is_saving{}; + uint8_t type; - if (typename Archive::is_saving()) - type = na.get_type_id(); + if (is_saving) + type = uint8_t(na.get_type_id()); a & type; - switch (type) + switch (epee::net_utils::address_type(type)) { - case epee::net_utils::ipv4_network_address::ID: - do_serialize(a, na, epee::net_utils::ipv4_network_address{0, 0}); - break; + case epee::net_utils::ipv4_network_address::get_type_id(): + do_serialize<epee::net_utils::ipv4_network_address>(is_saving, a, na); + break; + case net::tor_address::get_type_id(): + do_serialize<net::tor_address>(is_saving, a, na); + break; + case epee::net_utils::address_type::invalid: default: throw std::runtime_error("Unsupported network address type"); } @@ -76,6 +93,47 @@ namespace boost } template <class Archive, class ver_type> + inline void save(Archive& a, const net::tor_address& na, const ver_type) + { + const size_t length = std::strlen(na.host_str()); + if (length > 255) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + const uint16_t port{na.port()}; + const uint8_t len = length; + a & port; + a & len; + a.save_binary(na.host_str(), length); + } + + template <class Archive, class ver_type> + inline void load(Archive& a, net::tor_address& na, const ver_type) + { + uint16_t port = 0; + uint8_t length = 0; + a & port; + a & length; + + if (length > net::tor_address::buffer_size()) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + char host[net::tor_address::buffer_size()] = {0}; + a.load_binary(host, length); + host[sizeof(host) - 1] = 0; + + if (std::strcmp(host, net::tor_address::unknown_str()) == 0) + na = net::tor_address::unknown(); + else + na = MONERO_UNWRAP(net::tor_address::make(host, port)); + } + + template <class Archive, class ver_type> + inline void serialize(Archive &a, net::tor_address& na, const ver_type ver) + { + boost::serialization::split_free(a, na, ver); + } + + template <class Archive, class ver_type> inline void serialize(Archive &a, nodetool::peerlist_entry& pl, const ver_type ver) { a & pl.adr; diff --git a/src/p2p/p2p_protocol_defs.h b/src/p2p/p2p_protocol_defs.h index 939cedf42..6e5e45008 100644 --- a/src/p2p/p2p_protocol_defs.h +++ b/src/p2p/p2p_protocol_defs.h @@ -34,6 +34,7 @@ #include <boost/serialization/version.hpp> #include "serialization/keyvalue_serialization.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" // needed for serialization #include "misc_language.h" #include "string_tools.h" #include "time_helper.h" @@ -204,7 +205,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); @@ -263,7 +264,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); diff --git a/src/rpc/CMakeLists.txt b/src/rpc/CMakeLists.txt index d2c4a33cb..60cae036c 100644 --- a/src/rpc/CMakeLists.txt +++ b/src/rpc/CMakeLists.txt @@ -112,6 +112,7 @@ target_link_libraries(rpc common cryptonote_core cryptonote_protocol + net version ${Boost_REGEX_LIBRARY} ${Boost_THREAD_LIBRARY} diff --git a/src/rpc/core_rpc_server.cpp b/src/rpc/core_rpc_server.cpp index d308dd63d..b524273bf 100644 --- a/src/rpc/core_rpc_server.cpp +++ b/src/rpc/core_rpc_server.cpp @@ -42,6 +42,7 @@ using namespace epee; #include "cryptonote_basic/account.h" #include "cryptonote_basic/cryptonote_basic_impl.h" #include "misc_language.h" +#include "net/parse.h" #include "storages/http_abstract_invoke.h" #include "crypto/hash.h" #include "rpc/rpc_args.h" @@ -185,12 +186,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -902,12 +903,12 @@ namespace cryptonote PERF_TIMER(on_get_peer_list); std::vector<nodetool::peerlist_entry> white_list; std::vector<nodetool::peerlist_entry> gray_list; - m_p2p.get_peerlist_manager().get_peerlist_full(gray_list, white_list); + m_p2p.get_public_peerlist(gray_list, white_list); res.white_list.reserve(white_list.size()); for (auto & entry : white_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.white_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -917,7 +918,7 @@ namespace cryptonote res.gray_list.reserve(gray_list.size()); for (auto & entry : gray_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.gray_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -1646,12 +1647,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -1730,12 +1731,14 @@ namespace cryptonote epee::net_utils::network_address na; if (!i->host.empty()) { - if (!epee::net_utils::create_network_address(na, i->host)) + auto na_parsed = net::get_network_address(i->host, 0); + if (!na_parsed) { error_resp.code = CORE_RPC_ERROR_CODE_WRONG_PARAM; error_resp.message = "Unsupported host type"; return false; } + na = std::move(*na_parsed); } else { @@ -1958,11 +1961,7 @@ namespace cryptonote bool core_rpc_server::on_out_peers(const COMMAND_RPC_OUT_PEERS::request& req, COMMAND_RPC_OUT_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_out_peers); - size_t n_connections = m_p2p.get_outgoing_connections_count(); - size_t n_delete = (n_connections > req.out_peers) ? n_connections - req.out_peers : 0; - m_p2p.m_config.m_net_config.max_out_connection_count = req.out_peers; - if (n_delete) - m_p2p.delete_out_connections(n_delete); + m_p2p.change_max_out_public_peers(req.out_peers); res.status = CORE_RPC_STATUS_OK; return true; } @@ -1970,11 +1969,7 @@ namespace cryptonote bool core_rpc_server::on_in_peers(const COMMAND_RPC_IN_PEERS::request& req, COMMAND_RPC_IN_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_in_peers); - size_t n_connections = m_p2p.get_incoming_connections_count(); - size_t n_delete = (n_connections > req.in_peers) ? n_connections - req.in_peers : 0; - m_p2p.m_config.m_net_config.max_in_connection_count = req.in_peers; - if (n_delete) - m_p2p.delete_in_connections(n_delete); + m_p2p.change_max_in_public_peers(req.in_peers); res.status = CORE_RPC_STATUS_OK; return true; } diff --git a/src/rpc/daemon_handler.cpp b/src/rpc/daemon_handler.cpp index e2885dbb5..871f7d368 100644 --- a/src/rpc/daemon_handler.cpp +++ b/src/rpc/daemon_handler.cpp @@ -423,13 +423,13 @@ namespace rpc res.info.alt_blocks_count = chain.get_alternative_blocks_count(); - uint64_t total_conn = m_p2p.get_connections_count(); - res.info.outgoing_connections_count = m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = m_p2p.get_public_connections_count(); + res.info.outgoing_connections_count = m_p2p.get_public_outgoing_connections_count(); res.info.incoming_connections_count = total_conn - res.info.outgoing_connections_count; - res.info.white_peerlist_size = m_p2p.get_peerlist_manager().get_white_peers_count(); + res.info.white_peerlist_size = m_p2p.get_public_white_peers_count(); - res.info.grey_peerlist_size = m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.info.grey_peerlist_size = m_p2p.get_public_gray_peers_count(); res.info.mainnet = m_core.get_nettype() == MAINNET; res.info.testnet = m_core.get_nettype() == TESTNET; diff --git a/tests/unit_tests/CMakeLists.txt b/tests/unit_tests/CMakeLists.txt index 5f7fa84a5..cb421c847 100644 --- a/tests/unit_tests/CMakeLists.txt +++ b/tests/unit_tests/CMakeLists.txt @@ -62,6 +62,7 @@ set(unit_tests_sources mul_div.cpp multiexp.cpp multisig.cpp + net.cpp notify.cpp output_distribution.cpp parse_amount.cpp @@ -100,6 +101,7 @@ target_link_libraries(unit_tests cryptonote_core blockchain_db rpc + net serialization wallet p2p diff --git a/tests/unit_tests/epee_utils.cpp b/tests/unit_tests/epee_utils.cpp index 3d5882d7d..18fb262c2 100644 --- a/tests/unit_tests/epee_utils.cpp +++ b/tests/unit_tests/epee_utils.cpp @@ -545,6 +545,8 @@ TEST(StringTools, GetIpInt32) TEST(NetUtils, IPv4NetworkAddress) { + static_assert(epee::net_utils::ipv4_network_address::get_type_id() == epee::net_utils::address_type::ipv4, "bad ipv4 type id"); + const auto ip1 = boost::endian::native_to_big(0x330012FFu); const auto ip_loopback = boost::endian::native_to_big(0x7F000001u); const auto ip_local = boost::endian::native_to_big(0x0A000000u); @@ -555,7 +557,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip1, address1.ip()); EXPECT_EQ(65535, address1.port()); EXPECT_TRUE(epee::net_utils::ipv4_network_address{std::move(address1)} == address1); @@ -568,7 +570,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip_loopback, loopback.ip()); EXPECT_EQ(0, loopback.port()); @@ -624,7 +626,9 @@ TEST(NetUtils, NetworkAddress) constexpr static bool is_local() noexcept { return false; } static std::string str() { return {}; } static std::string host_str() { return {}; } - constexpr static uint8_t get_type_id() noexcept { return uint8_t(-1); } + constexpr static epee::net_utils::address_type get_type_id() noexcept { return epee::net_utils::address_type(-1); } + constexpr static epee::net_utils::zone get_zone() noexcept { return epee::net_utils::zone::invalid; } + constexpr static bool is_blockable() noexcept { return false; } }; const epee::net_utils::network_address empty; @@ -634,7 +638,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("<none>", empty.host_str().c_str()); EXPECT_FALSE(empty.is_loopback()); EXPECT_FALSE(empty.is_local()); - EXPECT_EQ(0, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::invalid, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::invalid, empty.get_zone()); + EXPECT_FALSE(empty.is_blockable()); EXPECT_THROW(empty.as<custom_address>(), std::bad_cast); epee::net_utils::network_address address1{ @@ -650,7 +656,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_TRUE(address1.is_blockable()); EXPECT_NO_THROW(address1.as<epee::net_utils::ipv4_network_address>()); EXPECT_THROW(address1.as<custom_address>(), std::bad_cast); @@ -667,7 +675,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); const epee::net_utils::network_address local{ epee::net_utils::ipv4_network_address{ip_local, 8080} diff --git a/tests/unit_tests/net.cpp b/tests/unit_tests/net.cpp new file mode 100644 index 000000000..a38ecfe81 --- /dev/null +++ b/tests/unit_tests/net.cpp @@ -0,0 +1,745 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include <atomic> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/asio/buffer.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/system/error_code.hpp> +#include <boost/thread/thread.hpp> +#include <cstring> +#include <functional> +#include <gtest/gtest.h> +#include <memory> + +#include "net/error.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "p2p/net_peerlist_boost_serialization.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" + +namespace +{ + static constexpr const char v2_onion[] = + "xmrto2bturnore26.onion"; + static constexpr const char v3_onion[] = + "vww6ybal4bd7szmgncyruucpgfkqahzddi37ktceo3ah7ngmcopnpyyd.onion"; +} + +TEST(tor_address, constants) +{ + static_assert(!net::tor_address::is_local(), "bad is_local() response"); + static_assert(!net::tor_address::is_loopback(), "bad is_loopback() response"); + static_assert(net::tor_address::get_type_id() == epee::net_utils::address_type::tor, "bad get_type_id() response"); + + EXPECT_FALSE(net::tor_address::is_local()); + EXPECT_FALSE(net::tor_address::is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); +} + +TEST(tor_address, invalid) +{ + EXPECT_TRUE(net::tor_address::make("").has_error()); + EXPECT_TRUE(net::tor_address::make(":").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion:").has_error()); + EXPECT_TRUE(net::tor_address::make(v2_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(v3_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v2_onion, sizeof(v2_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v3_onion, sizeof(v3_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":-").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":900a").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":65536").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":-1").has_error()); + + std::string onion{v3_onion}; + onion.at(10) = 1; + EXPECT_TRUE(net::tor_address::make(onion).has_error()); +} + +TEST(tor_address, unblockable_types) +{ + net::tor_address tor{}; + + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + tor = net::tor_address::unknown(); + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + EXPECT_EQ(net::tor_address{}, net::tor_address::unknown()); +} + +TEST(tor_address, valid) +{ + const auto address1 = net::tor_address::make(v3_onion); + + ASSERT_TRUE(address1.has_value()); + EXPECT_EQ(0u, address1->port()); + EXPECT_STREQ(v3_onion, address1->host_str()); + EXPECT_STREQ(v3_onion, address1->str().c_str()); + EXPECT_TRUE(address1->is_blockable()); + + net::tor_address address2{*address1}; + + EXPECT_EQ(0u, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_STREQ(v3_onion, address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_TRUE(address2.equal(*address1)); + EXPECT_TRUE(address1->equal(address2)); + EXPECT_TRUE(address2 == *address1); + EXPECT_TRUE(*address1 == address2); + EXPECT_FALSE(address2 != *address1); + EXPECT_FALSE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_FALSE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v2_onion} + ":6545")); + + EXPECT_EQ(6545, address2.port()); + EXPECT_STREQ(v2_onion, address2.host_str()); + EXPECT_EQ(std::string{v2_onion} + ":6545", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_FALSE(address2.is_same_host(*address1)); + EXPECT_FALSE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v3_onion} + ":", 65535)); + + EXPECT_EQ(65535, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_EQ(std::string{v3_onion} + ":65535", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); +} + +TEST(tor_address, generic_network_address) +{ + const epee::net_utils::network_address tor1{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address tor2{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address ip{epee::net_utils::ipv4_network_address{100, 200}}; + + EXPECT_EQ(tor1, tor2); + EXPECT_NE(ip, tor1); + EXPECT_LT(ip, tor1); + + EXPECT_STREQ(v3_onion, tor1.host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":8080", tor1.str()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor1.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor2.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::ipv4, ip.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor1.get_zone()); + EXPECT_EQ(epee::net_utils::zone::tor, tor2.get_zone()); + EXPECT_EQ(epee::net_utils::zone::public_, ip.get_zone()); + EXPECT_TRUE(tor1.is_blockable()); + EXPECT_TRUE(tor2.is_blockable()); + EXPECT_TRUE(ip.is_blockable()); +} + +namespace +{ + struct test_command + { + net::tor_address tor; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(tor); + END_KV_SERIALIZE_MAP() + }; +} + +TEST(tor_address, epee_serializev_v2) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v2_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v2_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serializev_v3) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v3_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v3_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serialize_unknown) +{ + std::string buffer{}; + { + test_command command{net::tor_address::unknown()}; + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(net::tor_address::unknown_str()), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, boost_serialize_v2) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v2_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_v3) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v3_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_unknown) +{ + std::string buffer{}; + { + const net::tor_address tor{}; + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); +} + +TEST(get_network_address, onion) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("onion", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address(".onion", 0); + EXPECT_EQ(net::error::invalid_tor_address, address); + + address = net::get_network_address(v3_onion, 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":1000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":2000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":65536", 1000); + EXPECT_EQ(net::error::invalid_port, address); +} + + +TEST(get_network_address, ipv4) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("0.0.0.", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.257", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.254", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("0.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("0.0.0.254:1000", address->str().c_str()); + + address = net::get_network_address("23.0.0.254:2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("23.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("23.0.0.254:2000", address->str().c_str()); +} + +namespace +{ + using stream_type = boost::asio::ip::tcp; + + struct io_thread + { + boost::asio::io_service io_service; + boost::asio::io_service::work work; + stream_type::socket server; + stream_type::acceptor acceptor; + boost::thread io; + std::atomic<bool> connected; + + io_thread() + : io_service(), + work(io_service), + server(io_service), + acceptor(io_service), + io([this] () { try { this->io_service.run(); } catch (const std::exception& e) { MERROR(e.what()); }}), + connected(false) + { + acceptor.open(boost::asio::ip::tcp::v4()); + acceptor.bind(stream_type::endpoint{boost::asio::ip::tcp::v4(), 0}); + acceptor.listen(); + acceptor.async_accept(server, [this] (boost::system::error_code error) { + this->connected = true; + if (error) + throw boost::system::system_error{error}; + }); + } + + ~io_thread() noexcept + { + io_service.stop(); + if (io.joinable()) + io.join(); + } + }; + + struct checked_client + { + std::atomic<bool>* called_; + bool expected_; + + void operator()(boost::system::error_code error, net::socks::client::stream_type::socket&&) const + { + EXPECT_EQ(expected_, bool(error)) << "Socks server: " << error.message(); + ASSERT_TRUE(called_ != nullptr); + (*called_) = true; + } + }; +} + +TEST(socks_client, unsupported_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_connect_command("example.com", 8080)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_resolve_command("example.com")); + EXPECT_TRUE(test_client->buffer().empty()); +} + +TEST(socks_client, no_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_FALSE(net::socks::client::send(std::move(test_client))); +} + +TEST(socks_client, connect_command) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, checked_client{std::addressof(called), false} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_connect_command("example.com", 8080)); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x1f, 0x90, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, connect_command_failed) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, checked_client{std::addressof(called), true} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE( + test_client->set_connect_command( + epee::net_utils::ipv4_network_address{boost::endian::native_to_big(std::uint32_t(5000)), 3000} + ) + ); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x0b, 0xb8, 0x00, 0x00, 0x13, 0x88, 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 91, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, resolve_command) +{ + static std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0xff, 0, 0xad, 0}; + + struct resolve_client : net::socks::client + { + std::atomic<unsigned> called_; + bool expected_; + + resolve_client(stream_type::socket&& proxy) + : net::socks::client(std::move(proxy), net::socks::version::v4a_tor) + , called_(0) + , expected_(false) + {}; + + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) override + { + EXPECT_EQ(this, self.get()); + EXPECT_EQ(expected_, bool(error)) << "Resolve failure: " << error.message(); + + if (!error) + { + ASSERT_EQ(sizeof(reply_bytes), buffer().size()); + EXPECT_EQ(0u, std::memcmp(buffer().data(), reply_bytes, sizeof(reply_bytes))); + } + + ++called_; + } + }; + + io_thread io{}; + stream_type::socket client{io.io_service}; + + auto test_client = std::make_shared<resolve_client>(std::move(client)); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(test_client, io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 0); + + test_client->expected_ = true; + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::send(test_client)); + + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + reply_bytes[1] = 91; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 1); +} + + diff --git a/tests/unit_tests/test_peerlist.cpp b/tests/unit_tests/test_peerlist.cpp index f638c6251..03aa48ea0 100644 --- a/tests/unit_tests/test_peerlist.cpp +++ b/tests/unit_tests/test_peerlist.cpp @@ -37,7 +37,7 @@ TEST(peer_list, peer_list_general) { nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); #define MAKE_IPV4_ADDRESS(a,b,c,d,e) epee::net_utils::ipv4_network_address{MAKE_IP(a,b,c,d),e} #define ADD_GRAY_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple; ple.last_seen=last_seen_;ple.adr = addr_; ple.id = id_;plm.append_with_peer_gray(ple);} #define ADD_WHITE_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple;ple.last_seen=last_seen_; ple.adr = addr_; ple.id = id_;plm.append_with_peer_white(ple);} @@ -77,9 +77,180 @@ TEST(peer_list, merge_peer_lists) //([^ \t]*)\t([^ \t]*):([^ \t]*) \tlast_seen: d(\d+)\.h(\d+)\.m(\d+)\.s(\d+)\n //ADD_NODE_TO_PL("\2", \3, 0x\1, (1353346618 -(\4*60*60*24+\5*60*60+\6*60+\7 )));\n nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); std::vector<nodetool::peerlist_entry> outer_bs; #define ADD_NODE_TO_PL(ip_, port_, id_, timestamp_) { nodetool::peerlist_entry ple; epee::string_tools::get_ip_int32_from_string(ple.adr.ip, ip_); ple.last_seen = timestamp_; ple.adr.port = port_; ple.id = id_;outer_bs.push_back(ple);} +} + +namespace +{ + bool check_empty(nodetool::peerlist_storage& peers, std::initializer_list<epee::net_utils::zone> zones) + { + bool pass = false; + for (const epee::net_utils::zone zone : zones) + { + const nodetool::peerlist_types types{peers.take_zone(zone)}; + EXPECT_TRUE(types.white.empty()); + EXPECT_TRUE(types.gray.empty()); + EXPECT_TRUE(types.anchor.empty()); + pass = (types.white.empty() && types.gray.empty() && types.anchor.empty()); + } + return pass; + } +} + +TEST(peerlist_storage, store) +{ + + using address_type = epee::net_utils::address_type; + using zone = epee::net_utils::zone; + + nodetool::peerlist_storage peers{}; + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + + std::string buffer{}; + { + nodetool::peerlist_types types{}; + types.white.push_back({epee::net_utils::ipv4_network_address{1000, 10}, 44, 55}); + types.white.push_back({net::tor_address::unknown(), 64, 75}); + types.gray.push_back({net::tor_address::unknown(), 99, 88}); + types.gray.push_back({epee::net_utils::ipv4_network_address{2000, 20}, 84, 45}); + types.anchor.push_back({epee::net_utils::ipv4_network_address{999, 654}, 444, 555}); + types.anchor.push_back({net::tor_address::unknown(), 14, 33}); + types.anchor.push_back({net::tor_address::unknown(), 24, 22}); + + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + nodetool::peerlist_types types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + { + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); + + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); } |