diff options
39 files changed, 4293 insertions, 826 deletions
diff --git a/ANONYMITY_NETWORKS.md b/ANONYMITY_NETWORKS.md new file mode 100644 index 000000000..6ac8cd999 --- /dev/null +++ b/ANONYMITY_NETWORKS.md @@ -0,0 +1,179 @@ +# Anonymity Networks with Monero + +Currently only Tor has been integrated into Monero. Providing support for +Kovri/I2P should be minimal, but has not yet been attempted. The usage of +these networks is still considered experimental - there are a few pessimistic +cases where privacy is leaked. The design is intended to maximize privacy of +the source of a transaction by broadcasting it over an anonymity network, while +relying on IPv4 for the remainder of messages to make surrounding node attacks +(via sybil) more difficult. + + +## Behavior + +If _any_ anonymity network is enabled, transactions being broadcast that lack +a valid "context" (i.e. the transaction did not come from a p2p connection), +will only be sent to peers on anonymity networks. If an anonymity network is +enabled but no peers over an anonymity network are available, an error is +logged and the transaction is kept for future broadcasting over an anonymity +network. The transaction will not be broadcast unless an anonymity connection +is made or until `monerod` is shutdown and restarted with only public +connections enabled. + + +## P2P Commands + +Only handshakes, peer timed syncs, and transaction broadcast messages are +supported over anonymity networks. If one `--add-exclusive-node` onion address +is specified, then no syncing will take place and only transaction broadcasting +can occur. It is therefore recommended that `--add-exclusive-node` be combined +with additional exclusive IPv4 address(es). + + +## Usage + +Anonymity networks have no seed nodes (the feature is still considered +experimental), so a user must specify an address. If configured properly, +additional peers can be found through typical p2p peerlist sharing. + +### Outbound Connections + +Connecting to an anonymous address requires the command line option +`--proxy` which tells `monerod` the ip/port of a socks proxy provided by a +separate process. On most systems the configuration will look like: + +> `--proxy tor,127.0.0.1:9050,10` +> `--proxy i2p,127.0.0.1:9000` + +which tells `monerod` that ".onion" p2p addresses can be forwarded to a socks +proxy at IP 127.0.0.1 port 9050 with a max of 10 outgoing connections and +".i2p" p2p addresses can be forwarded to a socks proxy at IP 127.0.0.1 port 9000 +with the default max outgoing connections. Since there are no seed nodes for +anonymity connections, peers must be manually specified: + +> `--add-exclusive-node rveahdfho7wo4b2m.onion:28083` +> `--add-peer rveahdfho7wo4b2m.onion:28083` + +Either option can be listed multiple times, and can specify any mix of Tor, +I2P, and IPv4 addresses. Using `--add-exclusive-node` will prevent the usage of +seed nodes on ALL networks, which will typically be undesireable. + +### Inbound Connections + +Receiving anonymity connections is done through the option +`--anonymous-inbound`. This option tells `monerod` the inbound address, network +type, and max connections: + +> `--anonymous-inbound rveahdfho7wo4b2m.onion:28083,127.0.0.28083,25` +> `--anonymous-inbound foobar.i2p:5000,127.0.0.1:30000` + +which tells `monerod` that a max of 25 inbound Tor connections are being +received at address "rveahdfho7wo4b2m.onion:28083" and forwarded to `monerod` +localhost port 28083, and a default max I2P connections are being received at +address "foobar.i2p:5000" and forwarded to `monerod` localhost port 30000. +These addresses will be shared with outgoing peers, over the same network type, +otherwise the peer will not be notified of the peer address by the proxy. + +### Network Types + +#### Tor + +Options `--add-exclusive-node` and `--add-peer` recognize ".onion" addresses, +and will properly forward those addresses to the proxy provided with +`--proxy tor,...`. + +Option `--anonymous-inbound` also recognizes ".onion" addresses, and will +automatically be sent out to outgoing Tor connections so the peer can +distribute the address to its other peers. + +##### Configuration + +Tor must be configured for hidden services. An example configuration ("torrc") +might look like: + +> HiddenServiceDir /var/lib/tor/data/monero +> HiddenServicePort 28083 127.0.0.1:28083 + +This will store key information in `/var/lib/tor/data/monero` and will forward +"Tor port" 28083 to port 28083 of ip 127.0.0.1. The file +`/usr/lib/tor/data/monero/hostname` will contain the ".onion" address for use +with `--anonymous-inbound`. + +#### Kovri/I2P + +Support for this network has not been implemented. Using ".i2p" addresses or +specifying "i2p" will currently generate an error. + + +## Privacy Limitations + +There are currently some techniques that could be used to _possibly_ identify +the machine that broadcast a transaction over an anonymity network. + +### Timestamps + +The peer timed sync command sends the current time in the message. This value +can be used to link an onion address to an IPv4/IPv6 address. If a peer first +sees a transaction over Tor, it could _assume_ (possibly incorrectly) that the +transaction originated from the peer. If both the Tor connection and an +IPv4/IPv6 connection have timestamps that are approximately close in value they +could be used to link the two connections. This is less likely to happen if the +system clock is fairly accurate - many peers on the Monero network should have +similar timestamps. + +#### Mitigation + +Keep the system clock accurate so that fingerprinting is more difficult. In +the future a random offset might be applied to anonymity networks so that if +the system clock is noticeably off (and therefore more fingerprintable), +linking the public IPv4/IPv6 connections with the anonymity networks will be +more difficult. + +### Bandwidth Usage + +An ISP can passively monitor `monerod` connections from a node and observe when +a transaction is sent over a Tor/Kovri connection via timing analysis + size of +data sent during that timeframe. Kovri should provide better protection against +this attack - its connections are not circuit based. However, if a node is +only using Kovri for broadcasting Monero transactions, the total aggregate of +Kovri/I2P data would also leak information. + +#### Mitigation + +There is no current mitigation for the user right now. This attack is fairly +sophisticated, and likely requires support from the internet host of a Monero +user. + +In the near future, "whitening" the amount of data sent over anonymity network +connections will be performed. An attempt will be made to make a transaction +broadcast indistinguishable from a peer timed sync command. + +### Intermittent Monero Syncing + +If a user only runs `monerod` to send a transaction then quit, this can also +be used by an ISP to link a user to a transaction. + +#### Mitigation + +Run `monerod` as often as possible to conceal when transactions are being sent. +Future versions will also have peers that first receive a transaction over an +anonymity network delay the broadcast to public peers by a randomized amount. +This will not completetely mitigate a user who syncs up sends then quits, in +part because this rule is not enforceable, so this mitigation strategy is +simply a best effort attempt. + +### Active Bandwidth Shaping + +An attacker could attempt to bandwidth shape traffic in an attempt to determine +the source of a Tor/Kovri/I2P connection. There isn't great mitigation against +this, but Kovri/I2P should provide better protection against this attack since +the connections are not circuit based. + +#### Mitigation + +The best mitigiation is to use Kovri/I2P instead of Tor. However, Kovri/I2P +has a smaller set of users (less cover traffic) and academic reviews, so there +is a tradeoff in potential isses. Also, anyone attempting this strategy really +wants to uncover a user, it seems unlikely that this would be performed against +every Tor/Kovri/I2P user. + @@ -615,6 +615,12 @@ See [README.i18n.md](README.i18n.md). ## Using Tor +> There is a new, still experimental, [integration with Tor](ANONYMITY_NETWORKS.md). The +> feature allows connecting over IPv4 and Tor simulatenously - IPv4 is used for +> relaying blocks and relaying transactions received by peers whereas Tor is +> used solely for relaying transactions received over local RPC. This provides +> privacy and better protection against surrounding node (sybil) attacks. + While Monero isn't made to integrate with Tor, it can be used wrapped with torsocks, by setting the following configuration parameters and environment variables: diff --git a/contrib/epee/include/net/abstract_tcp_server2.h b/contrib/epee/include/net/abstract_tcp_server2.h index e6b2755af..37f4c782d 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.h +++ b/contrib/epee/include/net/abstract_tcp_server2.h @@ -41,6 +41,7 @@ #include <boost/noncopyable.hpp> #include <boost/shared_ptr.hpp> #include <atomic> +#include <cassert> #include <map> #include <memory> @@ -87,14 +88,25 @@ namespace net_utils { public: typedef typename t_protocol_handler::connection_context t_connection_context; + + struct shared_state : socket_stats + { + shared_state() + : socket_stats(), pfilter(nullptr), config() + {} + + i_connection_filter* pfilter; + typename t_protocol_handler::config_type config; + }; + /// Construct a connection with the given io_service. - explicit connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter * &pfilter - ,t_connection_type connection_type); + boost::shared_ptr<shared_state> state, + t_connection_type connection_type); + + explicit connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type); virtual ~connection() noexcept(false); /// Get the socket associated with the connection. @@ -103,6 +115,9 @@ namespace net_utils /// Start the first asynchronous operation for the connection. bool start(bool is_income, bool is_multithreaded); + // `real_remote` is the actual endpoint (if connection is to proxy, etc.) + bool start(bool is_income, bool is_multithreaded, network_address real_remote); + void get_context(t_connection_context& context_){context_ = context;} void call_back_starter(); @@ -148,7 +163,6 @@ namespace net_utils //boost::array<char, 1024> buffer_; t_connection_context context; - i_connection_filter* &m_pfilter; // TODO what do they mean about wait on destructor?? --rfree : //this should be the last one, because it could be wait on destructor, while other activities possible on other threads @@ -210,7 +224,9 @@ namespace net_utils /// Stop the server. void send_stop_signal(); - bool is_stop_signal_sent(); + bool is_stop_signal_sent() const noexcept { return m_stop_signal_sent; }; + + const std::atomic<bool>& get_stop_signal() const noexcept { return m_stop_signal_sent; } void set_threads_prefix(const std::string& prefix_name); @@ -220,17 +236,28 @@ namespace net_utils void set_connection_filter(i_connection_filter* pfilter); + void set_default_remote(epee::net_utils::network_address remote) + { + default_remote = std::move(remote); + } + + bool add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote); bool connect(const std::string& adr, const std::string& port, uint32_t conn_timeot, t_connection_context& cn, const std::string& bind_ip = "0.0.0.0"); template<class t_callback> bool connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeot, const t_callback &cb, const std::string& bind_ip = "0.0.0.0"); - typename t_protocol_handler::config_type& get_config_object(){return m_config;} + typename t_protocol_handler::config_type& get_config_object() + { + assert(m_state != nullptr); // always set in constructor + return m_state->config; + } int get_binded_port(){return m_port;} long get_connections_count() const { - auto connections_count = (m_sock_count > 0) ? (m_sock_count - 1) : 0; // Socket count minus listening socket + assert(m_state != nullptr); // always set in constructor + auto connections_count = m_state->sock_count > 0 ? (m_state->sock_count - 1) : 0; // Socket count minus listening socket return connections_count; } @@ -292,9 +319,6 @@ namespace net_utils return true; } - protected: - typename t_protocol_handler::config_type m_config; - private: /// Run the server's io_service loop. bool worker_thread(); @@ -303,21 +327,21 @@ namespace net_utils bool is_thread_worker(); + const boost::shared_ptr<typename connection<t_protocol_handler>::shared_state> m_state; + /// The io_service used to perform asynchronous operations. std::unique_ptr<boost::asio::io_service> m_io_service_local_instance; boost::asio::io_service& io_service_; /// Acceptor used to listen for incoming connections. boost::asio::ip::tcp::acceptor acceptor_; + epee::net_utils::network_address default_remote; std::atomic<bool> m_stop_signal_sent; uint32_t m_port; - std::atomic<long> m_sock_count; - std::atomic<long> m_sock_number; std::string m_address; std::string m_thread_name_prefix; //TODO: change to enum server_type, now used size_t m_threads_count; - i_connection_filter* m_pfilter; std::vector<boost::shared_ptr<boost::thread> > m_threads; boost::thread::id m_main_thread_id; critical_section m_threads_lock; diff --git a/contrib/epee/include/net/abstract_tcp_server2.inl b/contrib/epee/include/net/abstract_tcp_server2.inl index 457ee2dd4..9c89a18cf 100644 --- a/contrib/epee/include/net/abstract_tcp_server2.inl +++ b/contrib/epee/include/net/abstract_tcp_server2.inl @@ -40,6 +40,7 @@ #include <boost/asio/deadline_timer.hpp> #include <boost/date_time/posix_time/posix_time.hpp> // TODO #include <boost/thread/condition_variable.hpp> // TODO +#include <boost/make_shared.hpp> #include "warnings.h" #include "string_tools.h" #include "misc_language.h" @@ -62,6 +63,13 @@ namespace epee { namespace net_utils { + template<typename T> + T& check_and_get(boost::shared_ptr<T>& ptr) + { + CHECK_AND_ASSERT_THROW_MES(bool(ptr), "shared_state cannot be null"); + return *ptr; + } + /************************************************************************/ /* */ /************************************************************************/ @@ -69,25 +77,31 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> connection<t_protocol_handler>::connection( boost::asio::io_service& io_service, - typename t_protocol_handler::config_type& config, - std::atomic<long> &ref_sock_count, // the ++/-- counter - std::atomic<long> &sock_number, // the only increasing ++ number generator - i_connection_filter* &pfilter - ,t_connection_type connection_type + boost::shared_ptr<shared_state> state, + t_connection_type connection_type + ) + : connection(boost::asio::ip::tcp::socket{io_service}, std::move(state), connection_type) + { + } + + template<class t_protocol_handler> + connection<t_protocol_handler>::connection( boost::asio::ip::tcp::socket&& sock, + boost::shared_ptr<shared_state> state, + t_connection_type connection_type ) : - connection_basic(io_service, ref_sock_count, sock_number), - m_protocol_handler(this, config, context), - m_pfilter( pfilter ), + connection_basic(std::move(sock), state), + m_protocol_handler(this, check_and_get(state).config, context), m_connection_type( connection_type ), m_throttle_speed_in("speed_in", "throttle_speed_in"), m_throttle_speed_out("speed_out", "throttle_speed_out"), - m_timer(io_service), + m_timer(socket_.get_io_service()), m_local(false), m_ready_to_close(false) { MDEBUG("test, connection constructor set m_connection_type="<<m_connection_type); } + PRAGMA_WARNING_DISABLE_VS(4355) //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -127,34 +141,44 @@ PRAGMA_WARNING_DISABLE_VS(4355) { TRY_ENTRY(); + boost::system::error_code ec; + auto remote_ep = socket_.remote_endpoint(ec); + CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); + CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); + + const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; + return start(is_income, is_multithreaded, ipv4_network_address{uint32_t(ip_), remote_ep.port()}); + CATCH_ENTRY_L0("connection<t_protocol_handler>::start()", false); + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> + bool connection<t_protocol_handler>::start(bool is_income, bool is_multithreaded, network_address real_remote) + { + TRY_ENTRY(); + // Use safe_shared_from_this, because of this is public method and it can be called on the object being deleted auto self = safe_shared_from_this(); if(!self) return false; m_is_multithreaded = is_multithreaded; + m_local = real_remote.is_loopback() || real_remote.is_local(); - boost::system::error_code ec; - auto remote_ep = socket_.remote_endpoint(ec); - CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get remote endpoint: " << ec.message() << ':' << ec.value()); - CHECK_AND_NO_ASSERT_MES(remote_ep.address().is_v4(), false, "IPv6 not supported here"); + // create a random uuid, we don't need crypto strength here + const boost::uuids::uuid random_uuid = boost::uuids::random_generator()(); + + context = t_connection_context{}; + context.set_details(random_uuid, std::move(real_remote), is_income); + boost::system::error_code ec; auto local_ep = socket_.local_endpoint(ec); CHECK_AND_NO_ASSERT_MES(!ec, false, "Failed to get local endpoint: " << ec.message() << ':' << ec.value()); - context = boost::value_initialized<t_connection_context>(); - const unsigned long ip_{boost::asio::detail::socket_ops::host_to_network_long(remote_ep.address().to_v4().to_ulong())}; - m_local = epee::net_utils::is_ip_loopback(ip_) || epee::net_utils::is_ip_local(ip_); - - // create a random uuid, we don't need crypto strength here - const boost::uuids::uuid random_uuid = boost::uuids::random_generator()(); - - context.set_details(random_uuid, epee::net_utils::ipv4_network_address(ip_, remote_ep.port()), is_income); _dbg3("[sock " << socket_.native_handle() << "] new connection from " << print_connection_context_short(context) << " to " << local_ep.address().to_string() << ':' << local_ep.port() << - ", total sockets objects " << m_ref_sock_count); + ", total sockets objects " << get_stats().sock_count); - if(m_pfilter && !m_pfilter->is_remote_host_allowed(context.m_remote_address)) + if(static_cast<shared_state&>(get_stats()).pfilter && !static_cast<shared_state&>(get_stats()).pfilter->is_remote_host_allowed(context.m_remote_address)) { _dbg2("[sock " << socket_.native_handle() << "] host denied " << context.m_remote_address.host_str() << ", shutdowning connection"); close(); @@ -279,7 +303,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) } MDEBUG(" connection type " << to_string( m_connection_type ) << " " << socket_.local_endpoint().address().to_string() << ":" << socket_.local_endpoint().port() - << " <--> " << address << ":" << port); + << " <--> " << context.m_remote_address.str() << " (via " << address << ":" << port << ")"); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -784,12 +808,14 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server( t_connection_type connection_type ) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), m_io_service_local_instance(new boost::asio::io_service()), io_service_(*m_io_service_local_instance.get()), acceptor_(io_service_), + default_remote(), m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + m_threads_count(0), + m_thread_index(0), m_connection_type( connection_type ), new_connection_() { @@ -799,11 +825,13 @@ PRAGMA_WARNING_DISABLE_VS(4355) template<class t_protocol_handler> boosted_tcp_server<t_protocol_handler>::boosted_tcp_server(boost::asio::io_service& extarnal_io_service, t_connection_type connection_type) : + m_state(boost::make_shared<typename connection<t_protocol_handler>::shared_state>()), io_service_(extarnal_io_service), acceptor_(io_service_), - m_stop_signal_sent(false), m_port(0), - m_sock_count(0), m_sock_number(0), m_threads_count(0), - m_pfilter(NULL), m_thread_index(0), + default_remote(), + m_stop_signal_sent(false), m_port(0), + m_threads_count(0), + m_thread_index(0), m_connection_type(connection_type), new_connection_() { @@ -844,7 +872,7 @@ PRAGMA_WARNING_DISABLE_VS(4355) boost::asio::ip::tcp::endpoint binded_endpoint = acceptor_.local_endpoint(); m_port = binded_endpoint.port(); MDEBUG("start accept"); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -922,7 +950,8 @@ POP_WARNINGS template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::set_connection_filter(i_connection_filter* pfilter) { - m_pfilter = pfilter; + assert(m_state != nullptr); // always set in constructor + m_state->pfilter = pfilter; } //--------------------------------------------------------------------------------- template<class t_protocol_handler> @@ -1030,12 +1059,6 @@ POP_WARNINGS } //--------------------------------------------------------------------------------- template<class t_protocol_handler> - bool boosted_tcp_server<t_protocol_handler>::is_stop_signal_sent() - { - return m_stop_signal_sent; - } - //--------------------------------------------------------------------------------- - template<class t_protocol_handler> void boosted_tcp_server<t_protocol_handler>::handle_accept(const boost::system::error_code& e) { MDEBUG("handle_accept"); @@ -1048,7 +1071,7 @@ POP_WARNINGS new_connection_->setRpcStation(); // hopefully this is not needed actually } connection_ptr conn(std::move(new_connection_)); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); @@ -1056,7 +1079,10 @@ POP_WARNINGS boost::asio::socket_base::keep_alive opt(true); conn->socket().set_option(opt); - conn->start(true, 1 < m_threads_count); + if (default_remote.get_type_id() == net_utils::address_type::invalid) + conn->start(true, 1 < m_threads_count); + else + conn->start(true, 1 < m_threads_count, default_remote); conn->save_dbg_log(); return; } @@ -1071,20 +1097,41 @@ POP_WARNINGS } // error path, if e or exception - _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("Some problems at accept: " << e.message() << ", connections_count = " << m_state->sock_count); misc_utils::sleep_no_w(100); - new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type)); + new_connection_.reset(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type)); acceptor_.async_accept(new_connection_->socket(), boost::bind(&boosted_tcp_server<t_protocol_handler>::handle_accept, this, boost::asio::placeholders::error)); } //--------------------------------------------------------------------------------- template<class t_protocol_handler> + bool boosted_tcp_server<t_protocol_handler>::add_connection(t_connection_context& out, boost::asio::ip::tcp::socket&& sock, network_address real_remote) + { + if(std::addressof(get_io_service()) == std::addressof(sock.get_io_service())) + { + connection_ptr conn(new connection<t_protocol_handler>(std::move(sock), m_state, m_connection_type)); + if(conn->start(false, 1 < m_threads_count, std::move(real_remote))) + { + conn->get_context(out); + conn->save_dbg_log(); + return true; + } + } + else + { + MWARNING(out << " was not added, socket/io_service mismatch"); + } + return false; + } + //--------------------------------------------------------------------------------- + template<class t_protocol_handler> bool boosted_tcp_server<t_protocol_handler>::connect(const std::string& adr, const std::string& port, uint32_t conn_timeout, t_connection_context& conn_context, const std::string& bind_ip) { TRY_ENTRY(); - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) ); connections_mutex.lock(); connections_.insert(new_connection_l); MDEBUG("connections_ size now " << connections_.size()); @@ -1187,7 +1234,8 @@ POP_WARNINGS } else { - _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_sock_count); + assert(m_state != nullptr); // always set in constructor + _erro("[sock " << new_connection_l->socket().native_handle() << "] Failed to start connection, connections_count = " << m_state->sock_count); } new_connection_l->save_dbg_log(); @@ -1201,7 +1249,7 @@ POP_WARNINGS bool boosted_tcp_server<t_protocol_handler>::connect_async(const std::string& adr, const std::string& port, uint32_t conn_timeout, const t_callback &cb, const std::string& bind_ip) { TRY_ENTRY(); - connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_config, m_sock_count, m_sock_number, m_pfilter, m_connection_type) ); + connection_ptr new_connection_l(new connection<t_protocol_handler>(io_service_, m_state, m_connection_type) ); connections_mutex.lock(); connections_.insert(new_connection_l); MDEBUG("connections_ size now " << connections_.size()); diff --git a/contrib/epee/include/net/connection_basic.hpp b/contrib/epee/include/net/connection_basic.hpp index 9b6fc14a7..b1b271db9 100644 --- a/contrib/epee/include/net/connection_basic.hpp +++ b/contrib/epee/include/net/connection_basic.hpp @@ -55,6 +55,15 @@ namespace epee { namespace net_utils { + struct socket_stats + { + socket_stats() + : sock_count(0), sock_number(0) + {} + + std::atomic<long> sock_count; + std::atomic<long> sock_number; + }; /************************************************************************/ /* */ @@ -72,6 +81,8 @@ class connection_basic_pimpl; // PIMPL for this class std::string to_string(t_connection_type type); class connection_basic { // not-templated base class for rapid developmet of some code parts + // beware of removing const, net_utils::connection is sketchily doing a cast to prevent storing ptr twice + const boost::shared_ptr<socket_stats> m_stats; public: std::unique_ptr< connection_basic_pimpl > mI; // my Implementation @@ -86,13 +97,15 @@ class connection_basic { // not-templated base class for rapid developmet of som /// Socket for the connection. boost::asio::ip::tcp::socket socket_; - std::atomic<long> &m_ref_sock_count; // reference to external counter of existing sockets that we will ++/-- public: // first counter is the ++/-- count of current sockets, the other socket_number is only-increasing ++ number generator - connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number); + connection_basic(boost::asio::ip::tcp::socket&& socket, boost::shared_ptr<socket_stats> stats); virtual ~connection_basic() noexcept(false); + //! \return `socket_stats` object passed in construction (ptr never changes). + socket_stats& get_stats() noexcept { return *m_stats; /* verified in constructor */ } + // various handlers to be called from connection class: void do_send_handler_write(const void * ptr , size_t cb); void do_send_handler_write_from_queue(const boost::system::error_code& e, size_t cb , int q_len); // from handle_write, sending next part diff --git a/contrib/epee/include/net/enums.h b/contrib/epee/include/net/enums.h new file mode 100644 index 000000000..078a4b274 --- /dev/null +++ b/contrib/epee/include/net/enums.h @@ -0,0 +1,65 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +namespace epee +{ +namespace net_utils +{ + enum class address_type : std::uint8_t + { + // Do not change values, this will break serialization + invalid = 0, + ipv4 = 1, + ipv6 = 2, + i2p = 3, + tor = 4 + }; + + enum class zone : std::uint8_t + { + invalid = 0, + public_ = 1, // public is keyword + i2p = 2, + tor = 3 + }; + + // implementations in src/net_utils_base.cpp + + //! \return String name of zone or "invalid" on error. + const char* zone_to_string(zone value) noexcept; + + //! \return `zone` enum of `value` or `zone::invalid` on error. + zone zone_from_string(boost::string_ref value) noexcept; +} // net_utils +} // epee + diff --git a/contrib/epee/include/net/net_utils_base.h b/contrib/epee/include/net/net_utils_base.h index a9e458626..82f8a7fe8 100644 --- a/contrib/epee/include/net/net_utils_base.h +++ b/contrib/epee/include/net/net_utils_base.h @@ -33,6 +33,7 @@ #include <boost/asio/io_service.hpp> #include <typeinfo> #include <type_traits> +#include "enums.h" #include "serialization/keyvalue_serialization.h" #include "misc_log_ex.h" @@ -43,6 +44,11 @@ #define MAKE_IP( a1, a2, a3, a4 ) (a1|(a2<<8)|(a3<<16)|(a4<<24)) #endif +namespace net +{ + class tor_address; +} + namespace epee { namespace net_utils @@ -53,6 +59,10 @@ namespace net_utils uint16_t m_port; public: + constexpr ipv4_network_address() noexcept + : ipv4_network_address(0, 0) + {} + constexpr ipv4_network_address(uint32_t ip, uint16_t port) noexcept : m_ip(ip), m_port(port) {} @@ -67,9 +77,10 @@ namespace net_utils std::string host_str() const; bool is_loopback() const; bool is_local() const; - static constexpr uint8_t get_type_id() noexcept { return ID; } + static constexpr address_type get_type_id() noexcept { return address_type::ipv4; } + static constexpr zone get_zone() noexcept { return zone::public_; } + static constexpr bool is_blockable() noexcept { return true; } - static const uint8_t ID = 1; BEGIN_KV_SERIALIZE_MAP() KV_SERIALIZE(m_ip) KV_SERIALIZE(m_port) @@ -103,7 +114,9 @@ namespace net_utils virtual std::string host_str() const = 0; virtual bool is_loopback() const = 0; virtual bool is_local() const = 0; - virtual uint8_t get_type_id() const = 0; + virtual address_type get_type_id() const = 0; + virtual zone get_zone() const = 0; + virtual bool is_blockable() const = 0; }; template<typename T> @@ -131,7 +144,9 @@ namespace net_utils virtual std::string host_str() const override { return value.host_str(); } virtual bool is_loopback() const override { return value.is_loopback(); } virtual bool is_local() const override { return value.is_local(); } - virtual uint8_t get_type_id() const override { return value.get_type_id(); } + virtual address_type get_type_id() const override { return value.get_type_id(); } + virtual zone get_zone() const override { return value.get_zone(); } + virtual bool is_blockable() const override { return value.is_blockable(); } }; std::shared_ptr<interface> self; @@ -146,6 +161,23 @@ namespace net_utils throw std::bad_cast{}; return static_cast<implementation<Type_>*>(self_)->value; } + + template<typename T, typename t_storage> + bool serialize_addr(std::false_type, t_storage& stg, typename t_storage::hsection hparent) + { + T addr{}; + if (!epee::serialization::selector<false>::serialize(addr, stg, hparent, "addr")) + return false; + *this = std::move(addr); + return true; + } + + template<typename T, typename t_storage> + bool serialize_addr(std::true_type, t_storage& stg, typename t_storage::hsection hparent) const + { + return epee::serialization::selector<true>::serialize(as<T>(), stg, hparent, "addr"); + } + public: network_address() : self(nullptr) {} template<typename T> @@ -158,43 +190,32 @@ namespace net_utils std::string host_str() const { return self ? self->host_str() : "<none>"; } bool is_loopback() const { return self ? self->is_loopback() : false; } bool is_local() const { return self ? self->is_local() : false; } - uint8_t get_type_id() const { return self ? self->get_type_id() : 0; } + address_type get_type_id() const { return self ? self->get_type_id() : address_type::invalid; } + zone get_zone() const { return self ? self->get_zone() : zone::invalid; } + bool is_blockable() const { return self ? self->is_blockable() : false; } template<typename Type> const Type &as() const { return as_mutable<const Type>(); } BEGIN_KV_SERIALIZE_MAP() - uint8_t type = is_store ? this_ref.get_type_id() : 0; + // need to `#include "net/tor_address.h"` when serializing `network_address` + static constexpr std::integral_constant<bool, is_store> is_store_{}; + + std::uint8_t type = std::uint8_t(is_store ? this_ref.get_type_id() : address_type::invalid); if (!epee::serialization::selector<is_store>::serialize(type, stg, hparent_section, "type")) return false; - switch (type) + + switch (address_type(type)) { - case ipv4_network_address::ID: - { - if (!is_store) - { - const_cast<network_address&>(this_ref) = ipv4_network_address{0, 0}; - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - MDEBUG("Found as addr: " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as<ipv4_network_address>()")) - MDEBUG("Found as template as<ipv4_network_address>(): " << this_ref.str()); - else if (epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "template as_mutable<ipv4_network_address>()")) - MDEBUG("Found as template as_mutable<ipv4_network_address>(): " << this_ref.str()); - else - { - MWARNING("Address not found"); - return false; - } - } - else - { - auto &addr = this_ref.template as_mutable<ipv4_network_address>(); - if (!epee::serialization::selector<is_store>::serialize(addr, stg, hparent_section, "addr")) - return false; - } + case address_type::ipv4: + return this_ref.template serialize_addr<ipv4_network_address>(is_store_, stg, hparent_section); + case address_type::tor: + return this_ref.template serialize_addr<net::tor_address>(is_store_, stg, hparent_section); + case address_type::invalid: + default: break; - } - default: MERROR("Unsupported network address type: " << (unsigned)type); return false; } + + MERROR("Unsupported network address type: " << (unsigned)type); + return false; END_KV_SERIALIZE_MAP() }; @@ -211,8 +232,6 @@ namespace net_utils inline bool operator>=(const network_address& lhs, const network_address& rhs) { return !lhs.less(rhs); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port = 0); - /************************************************************************/ /* */ /************************************************************************/ @@ -250,7 +269,7 @@ namespace net_utils {} connection_context_base(): m_connection_id(), - m_remote_address(ipv4_network_address{0,0}), + m_remote_address(), m_is_income(false), m_started(time(NULL)), m_last_recv(0), diff --git a/contrib/epee/src/connection_basic.cpp b/contrib/epee/src/connection_basic.cpp index 7d145ee46..f5f9b59fe 100644 --- a/contrib/epee/src/connection_basic.cpp +++ b/contrib/epee/src/connection_basic.cpp @@ -103,7 +103,7 @@ namespace net_utils // connection_basic_pimpl // ================================================================================================ -connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name) { } +connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_throttle(name), m_peer_number(0) { } // ================================================================================================ // connection_basic @@ -113,27 +113,31 @@ connection_basic_pimpl::connection_basic_pimpl(const std::string &name) : m_thro int connection_basic_pimpl::m_default_tos; // methods: -connection_basic::connection_basic(boost::asio::io_service& io_service, std::atomic<long> &ref_sock_count, std::atomic<long> &sock_number) - : +connection_basic::connection_basic(boost::asio::ip::tcp::socket&& socket, boost::shared_ptr<socket_stats> stats) + : + m_stats(std::move(stats)), mI( new connection_basic_pimpl("peer") ), - strand_(io_service), - socket_(io_service), + strand_(socket.get_io_service()), + socket_(std::move(socket)), m_want_close_connection(false), - m_was_shutdown(false), - m_ref_sock_count(ref_sock_count) -{ - ++ref_sock_count; // increase the global counter - mI->m_peer_number = sock_number.fetch_add(1); // use, and increase the generated number + m_was_shutdown(false) +{ + // add nullptr checks if removed + CHECK_AND_ASSERT_THROW_MES(bool(m_stats), "stats shared_ptr cannot be null"); + + ++(m_stats->sock_count); // increase the global counter + mI->m_peer_number = m_stats->sock_number.fetch_add(1); // use, and increase the generated number std::string remote_addr_str = "?"; try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; - _note("Spawned connection p2p#"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_ref_sock_count); + _note("Spawned connection p2p#"<<mI->m_peer_number<<" to " << remote_addr_str << " currently we have sockets count:" << m_stats->sock_count); } connection_basic::~connection_basic() noexcept(false) { + --(m_stats->sock_count); + std::string remote_addr_str = "?"; - m_ref_sock_count--; try { boost::system::error_code e; remote_addr_str = socket_.remote_endpoint(e).address().to_string(); } catch(...){} ; _note("Destructing connection p2p#"<<mI->m_peer_number << " to " << remote_addr_str); } diff --git a/contrib/epee/src/net_utils_base.cpp b/contrib/epee/src/net_utils_base.cpp index 263b344b4..9b781027e 100644 --- a/contrib/epee/src/net_utils_base.cpp +++ b/contrib/epee/src/net_utils_base.cpp @@ -8,8 +8,6 @@ namespace epee { namespace net_utils { - const uint8_t ipv4_network_address::ID; - bool ipv4_network_address::equal(const ipv4_network_address& other) const noexcept { return is_same_host(other) && port() == other.port(); } @@ -58,20 +56,6 @@ namespace epee { namespace net_utils return self_->is_same_host(*other_self); } - bool create_network_address(network_address &address, const std::string &string, uint16_t default_port) - { - uint32_t ip; - uint16_t port; - if (epee::string_tools::parse_peer_from_string(ip, port, string)) - { - if (default_port && !port) - port = default_port; - address = ipv4_network_address{ip, port}; - return true; - } - return false; - } - std::string print_connection_context(const connection_context_base& ctx) { std::stringstream ss; @@ -86,5 +70,31 @@ namespace epee { namespace net_utils return ss.str(); } + const char* zone_to_string(zone value) noexcept + { + switch (value) + { + case zone::public_: + return "public"; + case zone::i2p: + return "i2p"; + case zone::tor: + return "tor"; + default: + break; + } + return "invalid"; + } + + zone zone_from_string(const boost::string_ref value) noexcept + { + if (value == "public") + return zone::public_; + if (value == "i2p") + return zone::i2p; + if (value == "tor") + return zone::tor; + return zone::invalid; + } }} diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index a0b62da77..f1454b48e 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -110,6 +110,7 @@ add_subdirectory(checkpoints) add_subdirectory(cryptonote_basic) add_subdirectory(cryptonote_core) add_subdirectory(multisig) +add_subdirectory(net) if(NOT IOS) add_subdirectory(blockchain_db) endif() diff --git a/src/cryptonote_config.h b/src/cryptonote_config.h index 4f652cd42..93db71705 100644 --- a/src/cryptonote_config.h +++ b/src/cryptonote_config.h @@ -108,6 +108,7 @@ #define P2P_DEFAULT_PACKET_MAX_SIZE 50000000 //50000000 bytes maximum packet size #define P2P_DEFAULT_PEERS_IN_HANDSHAKE 250 #define P2P_DEFAULT_CONNECTION_TIMEOUT 5000 //5 seconds +#define P2P_DEFAULT_TOR_CONNECT_TIMEOUT 20 // seconds #define P2P_DEFAULT_PING_CONNECTION_TIMEOUT 2000 //2 seconds #define P2P_DEFAULT_INVOKE_TIMEOUT 60*2*1000 //2 minutes #define P2P_DEFAULT_HANDSHAKE_INVOKE_TIMEOUT 5000 //5 seconds diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.h b/src/cryptonote_protocol/cryptonote_protocol_handler.h index a1bd9171c..efd986b53 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.h +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.h @@ -173,15 +173,6 @@ namespace cryptonote //handler_response_blocks_now(blob.size()); // XXX return m_p2p->invoke_notify_to_peer(t_parameter::ID, epee::strspan<uint8_t>(blob), context); } - - template<class t_parameter> - bool relay_post_notify(typename t_parameter::request& arg, cryptonote_connection_context& exclude_context) - { - LOG_PRINT_L2("[" << epee::net_utils::print_connection_context_short(exclude_context) << "] post relay " << typeid(t_parameter).name() << " -->"); - std::string arg_buff; - epee::serialization::store_t_to_binary(arg, arg_buff); - return m_p2p->relay_notify_to_all(t_parameter::ID, epee::strspan<uint8_t>(arg_buff), exclude_context); - } }; } // namespace diff --git a/src/cryptonote_protocol/cryptonote_protocol_handler.inl b/src/cryptonote_protocol/cryptonote_protocol_handler.inl index 61a211094..c1459cbb6 100644 --- a/src/cryptonote_protocol/cryptonote_protocol_handler.inl +++ b/src/cryptonote_protocol/cryptonote_protocol_handler.inl @@ -226,7 +226,7 @@ namespace cryptonote cnx.host = cntxt.m_remote_address.host_str(); cnx.ip = ""; cnx.port = ""; - if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (cntxt.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { cnx.ip = cnx.host; cnx.port = std::to_string(cntxt.m_remote_address.as<epee::net_utils::ipv4_network_address>().port()); @@ -333,6 +333,13 @@ namespace cryptonote return true; } + // No chain synchronization over hidden networks (tor, i2p, etc.) + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + { + context.m_state = cryptonote_connection_context::state_normal; + return true; + } + if (hshd.current_height > target) { /* As I don't know if accessing hshd from core could be a good practice, @@ -2058,20 +2065,20 @@ skip: fluffy_arg.b.txs = fluffy_txs; // sort peers between fluffy ones and others - std::list<boost::uuids::uuid> fullConnections, fluffyConnections; + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> fullConnections, fluffyConnections; m_p2p->for_each_connection([this, &exclude_context, &fullConnections, &fluffyConnections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) { - if (peer_id && exclude_context.m_connection_id != context.m_connection_id) + if (peer_id && exclude_context.m_connection_id != context.m_connection_id && context.m_remote_address.get_zone() == epee::net_utils::zone::public_) { if(m_core.fluffy_blocks_enabled() && (support_flags & P2P_SUPPORT_FLAG_FLUFFY_BLOCKS)) { LOG_DEBUG_CC(context, "PEER SUPPORTS FLUFFY BLOCKS - RELAYING THIN/COMPACT WHATEVER BLOCK"); - fluffyConnections.push_back(context.m_connection_id); + fluffyConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } else { LOG_DEBUG_CC(context, "PEER DOESN'T SUPPORT FLUFFY BLOCKS - RELAYING FULL BLOCK"); - fullConnections.push_back(context.m_connection_id); + fullConnections.push_back({context.m_remote_address.get_zone(), context.m_connection_id}); } } return true; @@ -2082,13 +2089,13 @@ skip: { std::string fluffyBlob; epee::serialization::store_t_to_binary(fluffy_arg, fluffyBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), fluffyConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_FLUFFY_BLOCK::ID, epee::strspan<uint8_t>(fluffyBlob), std::move(fluffyConnections)); } if (!fullConnections.empty()) { std::string fullBlob; epee::serialization::store_t_to_binary(arg, fullBlob); - m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), fullConnections); + m_p2p->relay_notify_to_list(NOTIFY_NEW_BLOCK::ID, epee::strspan<uint8_t>(fullBlob), std::move(fullConnections)); } return true; @@ -2097,6 +2104,12 @@ skip: template<class t_core> bool t_cryptonote_protocol_handler<t_core>::relay_transactions(NOTIFY_NEW_TRANSACTIONS::request& arg, cryptonote_connection_context& exclude_context) { + const bool hide_tx_broadcast = + 1 < m_p2p->get_zone_count() && exclude_context.m_remote_address.get_zone() == epee::net_utils::zone::invalid; + + if (hide_tx_broadcast) + MDEBUG("Attempting to conceal origin of tx via anonymity network connection(s)"); + // no check for success, so tell core they're relayed unconditionally const bool pad_transactions = m_core.pad_transactions(); size_t bytes = pad_transactions ? 9 /* header */ + 4 /* 1 + 'txs' */ + tools::get_varint_data(arg.txs.size()).size() : 0; @@ -2131,7 +2144,30 @@ skip: // if the size of _ moved enough, we might lose byte in size encoding, we don't care } - return relay_post_notify<NOTIFY_NEW_TRANSACTIONS>(arg, exclude_context); + std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections; + m_p2p->for_each_connection([hide_tx_broadcast, &exclude_context, &connections](connection_context& context, nodetool::peerid_type peer_id, uint32_t support_flags) + { + const epee::net_utils::zone current_zone = context.m_remote_address.get_zone(); + const bool broadcast_to_peer = + peer_id && + (hide_tx_broadcast != bool(current_zone == epee::net_utils::zone::public_)) && + exclude_context.m_connection_id != context.m_connection_id; + + if (broadcast_to_peer) + connections.push_back({current_zone, context.m_connection_id}); + + return true; + }); + + if (connections.empty()) + MERROR("Transaction not relayed - no" << (hide_tx_broadcast ? " privacy": "") << " peers available"); + else + { + std::string fullBlob; + epee::serialization::store_t_to_binary(arg, fullBlob); + m_p2p->relay_notify_to_list(NOTIFY_NEW_TRANSACTIONS::ID, epee::strspan<uint8_t>(fullBlob), std::move(connections)); + } + return true; } //------------------------------------------------------------------------------------------------------------------------ template<class t_core> diff --git a/src/net/CMakeLists.txt b/src/net/CMakeLists.txt new file mode 100644 index 000000000..a81372125 --- /dev/null +++ b/src/net/CMakeLists.txt @@ -0,0 +1,34 @@ +# Copyright (c) 2018, The Monero Project +# +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without modification, are +# permitted provided that the following conditions are met: +# +# 1. Redistributions of source code must retain the above copyright notice, this list of +# conditions and the following disclaimer. +# +# 2. Redistributions in binary form must reproduce the above copyright notice, this list +# of conditions and the following disclaimer in the documentation and/or other +# materials provided with the distribution. +# +# 3. Neither the name of the copyright holder nor the names of its contributors may be +# used to endorse or promote products derived from this software without specific +# prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +# THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +# STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +# THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set(net_sources error.cpp parse.cpp socks.cpp tor_address.cpp) +set(net_headers error.h parse.h socks.h tor_address.h) + +monero_add_library(net ${net_sources} ${net_headers}) +target_link_libraries(net epee ${Boost_ASIO_LIBRARY}) + diff --git a/src/net/error.cpp b/src/net/error.cpp new file mode 100644 index 000000000..037f44d52 --- /dev/null +++ b/src/net/error.cpp @@ -0,0 +1,92 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "error.h" + +#include <string> + +namespace +{ + struct net_category : std::error_category + { + net_category() noexcept + : std::error_category() + {} + + const char* name() const noexcept override + { + return "net::error_category"; + } + + std::string message(int value) const override + { + switch (net::error(value)) + { + case net::error::expected_tld: + return "Expected top-level domain"; + case net::error::invalid_host: + return "Host value is not valid"; + case net::error::invalid_i2p_address: + return "Invalid I2P address"; + case net::error::invalid_port: + return "Invalid port value (expected 0-65535)"; + case net::error::invalid_tor_address: + return "Invalid Tor address"; + case net::error::unsupported_address: + return "Network address not supported"; + default: + break; + } + + return "Unknown net::error"; + } + + std::error_condition default_error_condition(int value) const noexcept override + { + switch (net::error(value)) + { + case net::error::invalid_port: + return std::errc::result_out_of_range; + case net::error::expected_tld: + case net::error::invalid_tor_address: + default: + break; + } + return std::error_condition{value, *this}; + } + }; +} // anonymous + +namespace net +{ + std::error_category const& error_category() noexcept + { + static const net_category instance{}; + return instance; + } +} diff --git a/src/net/error.h b/src/net/error.h new file mode 100644 index 000000000..c8338f7e2 --- /dev/null +++ b/src/net/error.h @@ -0,0 +1,64 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <system_error> +#include <type_traits> + +namespace net +{ + //! General net errors + enum class error : int + { + // 0 reserved for success (as per expect<T>) + expected_tld = 1, //!< Expected a tld + invalid_host, //!< Hostname is not valid + invalid_i2p_address, + invalid_port, //!< Outside of 0-65535 range + invalid_tor_address,//!< Invalid base32 or length + unsupported_address //!< Type not supported by `get_network_address` + }; + + //! \return `std::error_category` for `net` namespace. + std::error_category const& error_category() noexcept; + + //! \return `net::error` as a `std::error_code` value. + inline std::error_code make_error_code(error value) noexcept + { + return std::error_code{int(value), error_category()}; + } +} + +namespace std +{ + template<> + struct is_error_code_enum<::net::error> + : true_type + {}; +} diff --git a/src/net/fwd.h b/src/net/fwd.h new file mode 100644 index 000000000..ee7c539b0 --- /dev/null +++ b/src/net/fwd.h @@ -0,0 +1,45 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> + +namespace net +{ + enum class error : int; + class tor_address; + + namespace socks + { + class client; + template<typename> class connect_handler; + enum class error : int; + enum class version : std::uint8_t; + } +} diff --git a/src/net/parse.cpp b/src/net/parse.cpp new file mode 100644 index 000000000..ebf91eeff --- /dev/null +++ b/src/net/parse.cpp @@ -0,0 +1,60 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "parse.h" + +#include "net/tor_address.h" +#include "string_tools.h" + +namespace net +{ + expect<epee::net_utils::network_address> + get_network_address(const boost::string_ref address, const std::uint16_t default_port) + { + const boost::string_ref host = address.substr(0, address.rfind(':')); + + if (host.empty()) + return make_error_code(net::error::invalid_host); + if (host.ends_with(".onion")) + return tor_address::make(address, default_port); + if (host.ends_with(".i2p")) + return make_error_code(net::error::invalid_i2p_address); // not yet implemented (prevent public DNS lookup) + + std::uint16_t port = default_port; + if (host.size() < address.size()) + { + if (!epee::string_tools::get_xtype_from_string(port, std::string{address.substr(host.size() + 1)})) + return make_error_code(net::error::invalid_port); + } + + std::uint32_t ip = 0; + if (epee::string_tools::get_ip_int32_from_string(ip, std::string{host})) + return {epee::net_utils::ipv4_network_address{ip, port}}; + return make_error_code(net::error::unsupported_address); + } +} diff --git a/src/net/parse.h b/src/net/parse.h new file mode 100644 index 000000000..9195ddc2b --- /dev/null +++ b/src/net/parse.h @@ -0,0 +1,54 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> + +#include "common/expect.h" +#include "net/net_utils_base.h" + +namespace net +{ + /*! + Identifies onion and IPv4 addresses and returns them as a generic + `network_address`. If the type is unsupported, it might be a hostname, + and `error() == net::error::kUnsupportedAddress` is returned. + + \param address An onion address, ipv4 address or hostname. Hostname + will return an error. + \param default_port If `address` does not specify a port, this value + will be used. + + \return A tor or IPv4 address, else error. + */ + expect<epee::net_utils::network_address> + get_network_address(boost::string_ref address, std::uint16_t default_port); +} + diff --git a/src/net/socks.cpp b/src/net/socks.cpp new file mode 100644 index 000000000..f31efc8c1 --- /dev/null +++ b/src/net/socks.cpp @@ -0,0 +1,308 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "socks.h" + +#include <algorithm> +#include <boost/asio/buffer.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/arithmetic.hpp> +#include <boost/endian/conversion.hpp> +#include <cstring> +#include <limits> +#include <string> + +#include "net/net_utils_base.h" +#include "net/tor_address.h" + +namespace net +{ +namespace socks +{ + namespace + { + constexpr const unsigned v4_reply_size = 8; + constexpr const std::uint8_t v4_connect_command = 1; + constexpr const std::uint8_t v4tor_resolve_command = 0xf0; + constexpr const std::uint8_t v4_request_granted = 90; + + struct v4_header + { + std::uint8_t version; + std::uint8_t command_code; + boost::endian::big_uint16_t port; + boost::endian::big_uint32_t ip; + }; + + std::size_t write_domain_header(epee::span<std::uint8_t> out, const std::uint8_t command, const std::uint16_t port, const boost::string_ref domain) + { + if (std::numeric_limits<std::size_t>::max() - sizeof(v4_header) - 2 < domain.size()) + return 0; + + const std::size_t buf_size = sizeof(v4_header) + domain.size() + 2; + if (out.size() < buf_size) + return 0; + + // version 4, 1 indicates invalid ip for domain extension + const v4_header temp{4, command, port, std::uint32_t(1)}; + std::memcpy(out.data(), std::addressof(temp), sizeof(temp)); + out.remove_prefix(sizeof(temp)); + + *(out.data()) = 0; + out.remove_prefix(1); + + std::memcpy(out.data(), domain.data(), domain.size()); + out.remove_prefix(domain.size()); + + *(out.data()) = 0; + return buf_size; + } + + struct socks_category : boost::system::error_category + { + explicit socks_category() noexcept + : boost::system::error_category() + {} + + const char* name() const noexcept override + { + return "net::socks::error_category"; + } + + virtual std::string message(int value) const override + { + switch (socks::error(value)) + { + case socks::error::rejected: + return "Socks request rejected or failed"; + case socks::error::identd_connection: + return "Socks request rejected because server cannot connect to identd on the client"; + case socks::error::identd_user: + return "Socks request rejected because the client program and identd report different user-ids"; + + case socks::error::bad_read: + return "Socks boost::async_read read fewer bytes than expected"; + case socks::error::bad_write: + return "Socks boost::async_write wrote fewer bytes than expected"; + case socks::error::unexpected_version: + return "Socks server returned unexpected version in reply"; + + default: + break; + } + return "Unknown net::socks::error"; + } + + boost::system::error_condition default_error_condition(int value) const noexcept override + { + switch (socks::error(value)) + { + case socks::error::bad_read: + case socks::error::bad_write: + return boost::system::errc::io_error; + case socks::error::unexpected_version: + return boost::system::errc::protocol_error; + default: + break; + }; + if (1 <= value && value <= 256) + return boost::system::errc::protocol_error; + + return boost::system::error_condition{value, *this}; + } + }; + } + + const boost::system::error_category& error_category() noexcept + { + static const socks_category instance{}; + return instance; + } + + struct client::completed + { + std::shared_ptr<client> self_; + + void operator()(const boost::system::error_code error, const std::size_t bytes) const + { + static_assert(1 < sizeof(self_->buffer_), "buffer too small for v4 response"); + + if (self_) + { + client& self = *self_; + self.buffer_size_ = std::min(bytes, sizeof(self.buffer_)); + + if (error) + self.done(error, std::move(self_)); + else if (self.buffer().size() < sizeof(v4_header)) + self.done(socks::error::bad_read, std::move(self_)); + else if (self.buffer_[0] != 0) // response version + self.done(socks::error::unexpected_version, std::move(self_)); + else if (self.buffer_[1] != v4_request_granted) + self.done(socks::error(int(self.buffer_[1]) + 1), std::move(self_)); + else + self.done(boost::system::error_code{}, std::move(self_)); + } + } + }; + + struct client::read + { + std::shared_ptr<client> self_; + + static boost::asio::mutable_buffers_1 get_buffer(client& self) noexcept + { + static_assert(sizeof(v4_header) <= sizeof(self.buffer_), "buffer too small for v4 response"); + return boost::asio::buffer(self.buffer_, sizeof(v4_header)); + } + + void operator()(const boost::system::error_code error, const std::size_t bytes) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else if (bytes < self.buffer().size()) + self.done(socks::error::bad_write, std::move(self_)); + else + boost::asio::async_read(self.proxy_, get_buffer(self), completed{std::move(self_)}); + } + } + }; + + struct client::write + { + std::shared_ptr<client> self_; + + static boost::asio::const_buffers_1 get_buffer(client const& self) noexcept + { + return boost::asio::buffer(self.buffer_, self.buffer_size_); + } + + void operator()(const boost::system::error_code error) + { + if (self_) + { + client& self = *self_; + if (error) + self.done(error, std::move(self_)); + else + boost::asio::async_write(self.proxy_, get_buffer(self), read{std::move(self_)}); + } + } + }; + + client::client(stream_type::socket&& proxy, socks::version ver) + : proxy_(std::move(proxy)), buffer_size_(0), buffer_(), ver_(ver) + {} + + client::~client() {} + + bool client::set_connect_command(const epee::net_utils::ipv4_network_address& address) + { + switch (socks_version()) + { + case version::v4: + case version::v4a: + case version::v4a_tor: + break; + default: + return false; + } + + static_assert(sizeof(v4_header) < sizeof(buffer_), "buffer size too small for request"); + static_assert(0 < sizeof(buffer_), "buffer size too small for null termination"); + + // version 4 + const v4_header temp{4, v4_connect_command, address.port(), boost::endian::big_to_native(address.ip())}; + std::memcpy(std::addressof(buffer_), std::addressof(temp), sizeof(temp)); + buffer_[sizeof(temp)] = 0; + buffer_size_ = sizeof(temp) + 1; + + return true; + } + + bool client::set_connect_command(const boost::string_ref domain, std::uint16_t port) + { + switch (socks_version()) + { + case version::v4a: + case version::v4a_tor: + break; + + default: + return false; + } + + const std::size_t buf_used = write_domain_header(buffer_, v4_connect_command, port, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::set_connect_command(const net::tor_address& address) + { + if (!address.is_unknown()) + return set_connect_command(address.host_str(), address.port()); + return false; + } + + bool client::set_resolve_command(boost::string_ref domain) + { + if (socks_version() != version::v4a_tor) + return false; + + const std::size_t buf_used = write_domain_header(buffer_, v4tor_resolve_command, 0, domain); + buffer_size_ = buf_used; + return buf_used != 0; + } + + bool client::connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + alias.proxy_.async_connect(proxy_address, write{std::move(self)}); + return true; + } + return false; + } + + bool client::send(std::shared_ptr<client> self) + { + if (self && !self->buffer().empty()) + { + client& alias = *self; + boost::asio::async_write(alias.proxy_, write::get_buffer(alias), read{std::move(self)}); + return true; + } + return false; + } +} // socks +} // net diff --git a/src/net/socks.h b/src/net/socks.h new file mode 100644 index 000000000..d29a51ccb --- /dev/null +++ b/src/net/socks.h @@ -0,0 +1,225 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <cstdint> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/system/error_code.hpp> +#include <boost/type_traits/integral_constant.hpp> +#include <boost/utility/string_ref.hpp> +#include <memory> +#include <utility> + +#include "net/fwd.h" +#include "span.h" + +namespace epee +{ +namespace net_utils +{ + class ipv4_network_address; +} +} + +namespace net +{ +namespace socks +{ + //! Supported socks variants. + enum class version : std::uint8_t + { + v4 = 0, + v4a, + v4a_tor //!< Extensions defined in Tor codebase + }; + + //! Possible errors with socks communication. Defined in https://www.openssh.com/txt/socks4.protocol + enum class error : int + { + // 0 is reserved for success value + // 1-256 -> reserved for error values from socks server (+1 from wire value). + rejected = 92, + identd_connection, + identd_user, + // Specific to application + bad_read = 257, + bad_write, + unexpected_version + }; + + /* boost::system::error_code is extended for easier compatibility with + boost::asio errors. If std::error_code is needed (with expect<T> for + instance), then upgrade to boost 1.65+ or use conversion code in + develop branch at boost/system/detail/std_interoperability.hpp */ + + //! \return boost::system::error_category for net::socks namespace + const boost::system::error_category& error_category() noexcept; + + //! \return net::socks::error as a boost::system::error_code. + inline boost::system::error_code make_error_code(error value) noexcept + { + return boost::system::error_code{int(value), socks::error_category()}; + } + + //! Client support for socks connect and resolve commands. + class client + { + boost::asio::ip::tcp::socket proxy_; + std::uint16_t buffer_size_; + std::uint8_t buffer_[1024]; + socks::version ver_; + + /*! + Only invoked after `*send(...)` function completes or fails. + `bool(error) == false` indicates success; `self.get()` is always + `this` and allows implementations to skip + `std::enable_shared_from_this<T>` (ASIO callbacks need shared_ptr). + The design saves space and reduces cycles (everything uses moves, + so no atomic operations are ever necessary). + + \param error when processing last command (if any). + \param self `shared_ptr<client>` handle to `this`. + */ + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) = 0; + + public: + using stream_type = boost::asio::ip::tcp; + + // defined in cpp + struct write; + struct read; + struct completed; + + /*! + \param proxy ownership is passed into `this`. Does not have to be + in connected state. + \param ver socks version for the connection. + */ + explicit client(stream_type::socket&& proxy, socks::version ver); + + client(const client&) = delete; + virtual ~client(); + client& operator=(const client&) = delete; + + //! \return Ownership of socks client socket object. + stream_type::socket take_socket() + { + return stream_type::socket{std::move(proxy_)}; + } + + //! \return Socks version. + socks::version socks_version() const noexcept { return ver_; } + + //! \return Contents of internal buffer. + epee::span<const std::uint8_t> buffer() const noexcept + { + return {buffer_, buffer_size_}; + } + + //! \post `buffer.empty()`. + void clear_command() noexcept { buffer_size_ = 0; } + + //! Try to set `address` as remote connection request. + bool set_connect_command(const epee::net_utils::ipv4_network_address& address); + + //! Try to set `domain` + `port` as remote connection request. + bool set_connect_command(boost::string_ref domain, std::uint16_t port); + + //! Try to set `address` as remote Tor hidden service connection request. + bool set_connect_command(const net::tor_address& address); + + //! Try to set `domain` as remote DNS A record lookup request. + bool set_resolve_command(boost::string_ref domain); + + /*! + Asynchronously connect to `proxy_address` then issue command in + `buffer()`. The `done(...)` method will be invoked upon completion + with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + this function. + + \param self ownership of object is given to function. + \param proxy_address of the socks server. + \return False if `self->buffer().empty()` (no command set). + */ + static bool connect_and_send(std::shared_ptr<client> self, const stream_type::endpoint& proxy_address); + + /*! + Assume existing connection to proxy server; asynchronously issue + command in `buffer()`. The `done(...)` method will be invoked + upon completion with `self` and potential `error`s. + + \note Must use one of the `self->set_*_command` calls before using + the function. + + \param self ownership of object is given to function. + \return False if `self->buffer().empty()` (no command set). + */ + static bool send(std::shared_ptr<client> self); + }; + + template<typename Handler> + class connect_client : public client + { + Handler handler_; + + virtual void done(boost::system::error_code error, std::shared_ptr<client>) override + { + handler_(error, take_socket()); + } + + public: + explicit connect_client(stream_type::socket&& proxy, socks::version ver, Handler&& handler) + : client(std::move(proxy), ver), handler_(std::move(handler)) + {} + + virtual ~connect_client() override {} + }; + + template<typename Handler> + inline std::shared_ptr<client> + make_connect_client(client::stream_type::socket&& proxy, socks::version ver, Handler handler) + { + return std::make_shared<connect_client<Handler>>(std::move(proxy), ver, std::move(handler)); + } +} // socks +} // net + +namespace boost +{ +namespace system +{ + template<> + struct is_error_code_enum<net::socks::error> + : true_type + {}; +} // system +} // boost diff --git a/src/net/tor_address.cpp b/src/net/tor_address.cpp new file mode 100644 index 000000000..904a9a0fc --- /dev/null +++ b/src/net/tor_address.cpp @@ -0,0 +1,203 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "tor_address.h" + +#include <algorithm> +#include <boost/spirit/include/karma_generate.hpp> +#include <boost/spirit/include/karma_uint.hpp> +#include <cassert> +#include <cstring> +#include <limits> + +#include "net/error.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" +#include "string_tools.h" + +namespace net +{ + namespace + { + constexpr const char tld[] = u8".onion"; + constexpr const char unknown_host[] = "<unknown tor host>"; + + constexpr const unsigned v2_length = 16; + constexpr const unsigned v3_length = 56; + + constexpr const char base32_alphabet[] = + u8"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz234567"; + + expect<void> host_check(boost::string_ref host) noexcept + { + if (!host.ends_with(tld)) + return {net::error::expected_tld}; + + host.remove_suffix(sizeof(tld) - 1); + + //! \TODO v3 has checksum, base32 decoding is required to verify it + if (host.size() != v2_length && host.size() != v3_length) + return {net::error::invalid_tor_address}; + if (host.find_first_not_of(base32_alphabet) != boost::string_ref::npos) + return {net::error::invalid_tor_address}; + + return success(); + } + + struct tor_serialized + { + std::string host; + std::uint16_t port; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(host) + KV_SERIALIZE(port) + END_KV_SERIALIZE_MAP() + }; + } + + tor_address::tor_address(const boost::string_ref host, const std::uint16_t port) noexcept + : port_(port) + { + // this is a private constructor, throw if moved to public + assert(host.size() < sizeof(host_)); + + const std::size_t length = std::min(sizeof(host_) - 1, host.size()); + std::memcpy(host_, host.data(), length); + std::memset(host_ + length, 0, sizeof(host_) - length); + } + + const char* tor_address::unknown_str() noexcept + { + return unknown_host; + } + + tor_address::tor_address() noexcept + : port_(0) + { + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); + std::memset(host_ + sizeof(unknown_host), 0, sizeof(host_) - sizeof(unknown_host)); + } + + expect<tor_address> tor_address::make(const boost::string_ref address, const std::uint16_t default_port) + { + boost::string_ref host = address.substr(0, address.rfind(':')); + const boost::string_ref port = + address.substr(host.size() + (host.size() == address.size() ? 0 : 1)); + + MONERO_CHECK(host_check(host)); + + std::uint16_t porti = default_port; + if (!port.empty() && !epee::string_tools::get_xtype_from_string(porti, std::string{port})) + return {net::error::invalid_port}; + + static_assert(v2_length <= v3_length, "bad internal host size"); + static_assert(v3_length + sizeof(tld) == sizeof(tor_address::host_), "bad internal host size"); + return tor_address{host, porti}; + } + + bool tor_address::_load(epee::serialization::portable_storage& src, epee::serialization::section* hparent) + { + tor_serialized in{}; + if (in._load(src, hparent) && in.host.size() < sizeof(host_) && (in.host == unknown_host || !host_check(in.host).has_error())) + { + std::memcpy(host_, in.host.data(), in.host.size()); + std::memset(host_ + in.host.size(), 0, sizeof(host_) - in.host.size()); + port_ = in.port; + return true; + } + static_assert(sizeof(unknown_host) <= sizeof(host_), "bad buffer size"); + std::memcpy(host_, unknown_host, sizeof(unknown_host)); // include null terminator + port_ = 0; + return false; + } + + bool tor_address::store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const + { + const tor_serialized out{std::string{host_}, port_}; + return out.store(dest, hparent); + } + + tor_address::tor_address(const tor_address& rhs) noexcept + : port_(rhs.port_) + { + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + + tor_address& tor_address::operator=(const tor_address& rhs) noexcept + { + if (this != std::addressof(rhs)) + { + port_ = rhs.port_; + std::memcpy(host_, rhs.host_, sizeof(host_)); + } + return *this; + } + + bool tor_address::is_unknown() const noexcept + { + static_assert(1 <= sizeof(host_), "host size too small"); + return host_[0] == '<'; // character is not allowed otherwise + } + + bool tor_address::equal(const tor_address& rhs) const noexcept + { + return port_ == rhs.port_ && is_same_host(rhs); + } + + bool tor_address::less(const tor_address& rhs) const noexcept + { + return std::strcmp(host_str(), rhs.host_str()) < 0 || port() < rhs.port(); + } + + bool tor_address::is_same_host(const tor_address& rhs) const noexcept + { + //! \TODO v2 and v3 should be comparable - requires base32 + return std::strcmp(host_str(), rhs.host_str()) == 0; + } + + std::string tor_address::str() const + { + const std::size_t host_length = std::strlen(host_str()); + const std::size_t port_length = + port_ == 0 ? 0 : std::numeric_limits<std::uint16_t>::digits10 + 2; + + std::string out{}; + out.reserve(host_length + port_length); + out.assign(host_str(), host_length); + + if (port_ != 0) + { + out.push_back(':'); + namespace karma = boost::spirit::karma; + karma::generate(std::back_inserter(out), karma::ushort_, port()); + } + return out; + } +} diff --git a/src/net/tor_address.h b/src/net/tor_address.h new file mode 100644 index 000000000..22d8cc119 --- /dev/null +++ b/src/net/tor_address.h @@ -0,0 +1,140 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#pragma once + +#include <boost/utility/string_ref.hpp> +#include <cstdint> +#include <string> + +#include "common/expect.h" +#include "net/enums.h" +#include "net/error.h" + +namespace epee +{ +namespace serialization +{ + class portable_storage; + struct section; +} +} + +namespace net +{ + //! Tor onion address; internal format not condensed/decoded. + class tor_address + { + std::uint16_t port_; + char host_[63]; // null-terminated + + //! Keep in private, `host.size()` has no runtime check + tor_address(boost::string_ref host, std::uint16_t port) noexcept; + + public: + //! \return Size of internal buffer for host. + static constexpr std::size_t buffer_size() noexcept { return sizeof(host_); } + + //! \return `<unknown tor host>`. + static const char* unknown_str() noexcept; + + //! An object with `port() == 0` and `host_str() == unknown_str()`. + tor_address() noexcept; + + //! \return A default constructed `tor_address` object. + static tor_address unknown() noexcept { return tor_address{}; } + + /*! + Parse `address` in onion v2 or v3 format with (i.e. x.onion:80) + with `default_port` being used iff port is not specified in + `address`. + */ + static expect<tor_address> make(boost::string_ref address, std::uint16_t default_port = 0); + + //! Load from epee p2p format, and \return false if not valid tor address + bool _load(epee::serialization::portable_storage& src, epee::serialization::section* hparent); + + //! Store in epee p2p format + bool store(epee::serialization::portable_storage& dest, epee::serialization::section* hparent) const; + + // Moves and copies are currently identical + + tor_address(const tor_address& rhs) noexcept; + ~tor_address() = default; + tor_address& operator=(const tor_address& rhs) noexcept; + + //! \return True if default constructed or via `unknown()`. + bool is_unknown() const noexcept; + + bool equal(const tor_address& rhs) const noexcept; + bool less(const tor_address& rhs) const noexcept; + + //! \return True if onion addresses are identical. + bool is_same_host(const tor_address& rhs) const noexcept; + + //! \return `x.onion` or `x.onion:z` if `port() != 0`. + std::string str() const; + + //! \return Null-terminated `x.onion` value or `unknown_str()`. + const char* host_str() const noexcept { return host_; } + + //! \return Port value or `0` if unspecified. + std::uint16_t port() const noexcept { return port_; } + + static constexpr bool is_loopback() noexcept { return false; } + static constexpr bool is_local() noexcept { return false; } + + static constexpr epee::net_utils::address_type get_type_id() noexcept + { + return epee::net_utils::address_type::tor; + } + + static constexpr epee::net_utils::zone get_zone() noexcept + { + return epee::net_utils::zone::tor; + } + + //! \return `!is_unknown()`. + bool is_blockable() const noexcept { return !is_unknown(); } + }; + + inline bool operator==(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.equal(rhs); + } + + inline bool operator!=(const tor_address& lhs, const tor_address& rhs) noexcept + { + return !lhs.equal(rhs); + } + + inline bool operator<(const tor_address& lhs, const tor_address& rhs) noexcept + { + return lhs.less(rhs); + } +} // net diff --git a/src/p2p/CMakeLists.txt b/src/p2p/CMakeLists.txt index 9b924907e..9a1730b8a 100644 --- a/src/p2p/CMakeLists.txt +++ b/src/p2p/CMakeLists.txt @@ -40,6 +40,7 @@ target_link_libraries(p2p PUBLIC version cryptonote_core + net ${UPNP_LIBRARIES} ${Boost_CHRONO_LIBRARY} ${Boost_PROGRAM_OPTIONS_LIBRARY} diff --git a/src/p2p/net_node.cpp b/src/p2p/net_node.cpp index 7cad6e077..8639fdb3b 100644 --- a/src/p2p/net_node.cpp +++ b/src/p2p/net_node.cpp @@ -28,9 +28,79 @@ // // Parts of this file are originally copyright (c) 2012-2013 The Cryptonote developers +#include <boost/algorithm/string/find_iterator.hpp> +#include <boost/algorithm/string/finder.hpp> +#include <boost/chrono/duration.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/optional/optional.hpp> +#include <boost/thread/future.hpp> +#include <boost/utility/string_ref.hpp> +#include <chrono> +#include <utility> + #include "common/command_line.h" #include "cryptonote_core/cryptonote_core.h" +#include "cryptonote_protocol/cryptonote_protocol_defs.h" #include "net_node.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "p2p/p2p_protocol_defs.h" +#include "string_tools.h" + +namespace +{ + constexpr const boost::chrono::milliseconds future_poll_interval{500}; + constexpr const std::chrono::seconds tor_connect_timeout{P2P_DEFAULT_TOR_CONNECT_TIMEOUT}; + + std::int64_t get_max_connections(const boost::iterator_range<boost::string_ref::const_iterator> value) noexcept + { + // -1 is default, 0 is error + if (value.empty()) + return -1; + + std::uint32_t out = 0; + if (epee::string_tools::get_xtype_from_string(out, std::string{value.begin(), value.end()})) + return out; + return 0; + } + + template<typename T> + epee::net_utils::network_address get_address(const boost::string_ref value) + { + expect<T> address = T::make(value); + if (!address) + { + MERROR( + "Failed to parse " << epee::net_utils::zone_to_string(T::get_zone()) << " address \"" << value << "\": " << address.error().message() + ); + return {}; + } + return {std::move(*address)}; + } + + bool start_socks(std::shared_ptr<net::socks::client> client, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + CHECK_AND_ASSERT_MES(client != nullptr, false, "Unexpected null client"); + + bool set = false; + switch (remote.get_type_id()) + { + case net::tor_address::get_type_id(): + set = client->set_connect_command(remote.as<net::tor_address>()); + break; + default: + MERROR("Unsupported network address in socks_connect"); + return false; + } + + const bool sent = + set && net::socks::client::connect_and_send(std::move(client), proxy); + CHECK_AND_ASSERT_MES(sent, false, "Unexpected failure to init socks client"); + return true; + } +} namespace nodetool { @@ -55,6 +125,8 @@ namespace nodetool const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node = {"add-exclusive-node", "Specify list of peers to connect to only." " If this option is given the options add-priority-node and seed-node are ignored"}; const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node = {"seed-node", "Connect to a node to retrieve peer addresses, and disconnect"}; + const command_line::arg_descriptor<std::vector<std::string> > arg_proxy = {"proxy", "<network-type>,<socks-ip:port>[,max_connections] i.e. \"tor,127.0.0.1:9050,100\""}; + const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound = {"anonymous-inbound", "<hidden-service-address>,<[bind-ip:]port>[,max_connections] i.e. \"x.onion,127.0.0.1:18083,100\""}; const command_line::arg_descriptor<bool> arg_p2p_hide_my_port = {"hide-my-port", "Do not announce yourself as peerlist candidate", false, true}; const command_line::arg_descriptor<bool> arg_no_igd = {"no-igd", "Disable UPnP port mapping"}; @@ -67,4 +139,196 @@ namespace nodetool const command_line::arg_descriptor<int64_t> arg_limit_rate = {"limit-rate", "set limit-rate [kB/s]", -1}; const command_line::arg_descriptor<bool> arg_save_graph = {"save-graph", "Save data for dr monero", false}; + + boost::optional<std::vector<proxy>> get_proxies(boost::program_options::variables_map const& vm) + { + namespace ip = boost::asio::ip; + + std::vector<proxy> proxies{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_proxy); + proxies.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + proxies.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No network type for --" << arg_proxy.name); + const boost::string_ref zone{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No ipv4:port given for --" << arg_proxy.name); + const boost::string_ref proxy{next->begin(), next->size()}; + + ++next; + if (!next.eof()) + { + proxies.back().max_connections = get_max_connections(*next); + if (proxies.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + switch (epee::net_utils::zone_from_string(zone)) + { + case epee::net_utils::zone::tor: + proxies.back().zone = epee::net_utils::zone::tor; + break; + default: + MERROR("Invalid network for --" << arg_proxy.name); + return boost::none; + } + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{proxy}) || port == 0) + { + MERROR("Invalid ipv4:port given for --" << arg_proxy.name); + return boost::none; + } + proxies.back().address = ip::tcp::endpoint{ip::address_v4{boost::endian::native_to_big(ip)}, port}; + } + + return proxies; + } + + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(boost::program_options::variables_map const& vm) + { + std::vector<anonymous_inbound> inbounds{}; + + const std::vector<std::string> args = command_line::get_arg(vm, arg_anonymous_inbound); + inbounds.reserve(args.size()); + + for (const boost::string_ref arg : args) + { + inbounds.emplace_back(); + + auto next = boost::algorithm::make_split_iterator(arg, boost::algorithm::first_finder(",")); + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No inbound address for --" << arg_anonymous_inbound.name); + const boost::string_ref address{next->begin(), next->size()}; + + ++next; + CHECK_AND_ASSERT_MES(!next.eof() && !next->empty(), boost::none, "No local ipv4:port given for --" << arg_anonymous_inbound.name); + const boost::string_ref bind{next->begin(), next->size()}; + + const std::size_t colon = bind.find_first_of(':'); + CHECK_AND_ASSERT_MES(colon < bind.size(), boost::none, "No local port given for --" << arg_anonymous_inbound.name); + + ++next; + if (!next.eof()) + { + inbounds.back().max_connections = get_max_connections(*next); + if (inbounds.back().max_connections == 0) + { + MERROR("Invalid max connections given to --" << arg_proxy.name); + return boost::none; + } + } + + expect<epee::net_utils::network_address> our_address = net::get_network_address(address, 0); + switch (our_address ? our_address->get_type_id() : epee::net_utils::address_type::invalid) + { + case net::tor_address::get_type_id(): + inbounds.back().our_address = std::move(*our_address); + inbounds.back().default_remote = net::tor_address::unknown(); + break; + default: + MERROR("Invalid inbound address (" << address << ") for --" << arg_anonymous_inbound.name << ": " << (our_address ? "invalid type" : our_address.error().message())); + return boost::none; + } + + // get_address returns default constructed address on error + if (inbounds.back().our_address == epee::net_utils::network_address{}) + return boost::none; + + std::uint32_t ip = 0; + std::uint16_t port = 0; + if (!epee::string_tools::parse_peer_from_string(ip, port, std::string{bind})) + { + MERROR("Invalid ipv4:port given for --" << arg_anonymous_inbound.name); + return boost::none; + } + inbounds.back().local_ip = std::string{bind.substr(0, colon)}; + inbounds.back().local_port = std::string{bind.substr(colon + 1)}; + } + + return inbounds; + } + + bool is_filtered_command(const epee::net_utils::network_address& address, int command) + { + switch (command) + { + case nodetool::COMMAND_HANDSHAKE_T<cryptonote::CORE_SYNC_DATA>::ID: + case nodetool::COMMAND_TIMED_SYNC_T<cryptonote::CORE_SYNC_DATA>::ID: + case cryptonote::NOTIFY_NEW_TRANSACTIONS::ID: + return false; + default: + break; + } + + if (address.get_zone() == epee::net_utils::zone::public_) + return false; + + MWARNING("Filtered command (#" << command << ") to/from " << address.str()); + return true; + } + + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote) + { + using socket_type = net::socks::client::stream_type::socket; + using client_result = std::pair<boost::system::error_code, socket_type>; + + struct notify + { + boost::promise<client_result> socks_promise; + + void operator()(boost::system::error_code error, socket_type&& sock) + { + socks_promise.set_value(std::make_pair(error, std::move(sock))); + } + }; + + boost::unique_future<client_result> socks_result{}; + { + boost::promise<client_result> socks_promise{}; + socks_result = socks_promise.get_future(); + + auto client = net::socks::make_connect_client( + boost::asio::ip::tcp::socket{service}, net::socks::version::v4a, notify{std::move(socks_promise)} + ); + if (!start_socks(std::move(client), proxy, remote)) + return boost::none; + } + + const auto start = std::chrono::steady_clock::now(); + while (socks_result.wait_for(future_poll_interval) == boost::future_status::timeout) + { + if (tor_connect_timeout < std::chrono::steady_clock::now() - start) + { + MERROR("Timeout on socks connect (" << proxy << " to " << remote.str() << ")"); + return boost::none; + } + + if (stop_signal) + return boost::none; + } + + try + { + auto result = socks_result.get(); + if (!result.first) + return {std::move(result.second)}; + + MERROR("Failed to make socks connection to " << remote.str() << " (via " << proxy << "): " << result.first.message()); + } + catch (boost::broken_promise const&) + {} + + return boost::none; + } } diff --git a/src/p2p/net_node.h b/src/p2p/net_node.h index 09c219a44..112f30fb6 100644 --- a/src/p2p/net_node.h +++ b/src/p2p/net_node.h @@ -30,12 +30,17 @@ #pragma once #include <array> +#include <atomic> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> #include <boost/thread.hpp> +#include <boost/optional/optional_fwd.hpp> #include <boost/program_options/options_description.hpp> #include <boost/program_options/variables_map.hpp> -#include <boost/serialization/version.hpp> #include <boost/uuid/uuid.hpp> -#include <boost/serialization/map.hpp> +#include <functional> +#include <utility> +#include <vector> #include "cryptonote_config.h" #include "warnings.h" @@ -47,6 +52,8 @@ #include "net_peerlist.h" #include "math_helper.h" #include "net_node_common.h" +#include "net/enums.h" +#include "net/fwd.h" #include "common/command_line.h" PUSH_WARNINGS @@ -54,6 +61,47 @@ DISABLE_VS_WARNINGS(4355) namespace nodetool { + struct proxy + { + proxy() + : max_connections(-1), + address(), + zone(epee::net_utils::zone::invalid) + {} + + std::int64_t max_connections; + boost::asio::ip::tcp::endpoint address; + epee::net_utils::zone zone; + }; + + struct anonymous_inbound + { + anonymous_inbound() + : max_connections(-1), + local_ip(), + local_port(), + our_address(), + default_remote() + {} + + std::int64_t max_connections; + std::string local_ip; + std::string local_port; + epee::net_utils::network_address our_address; + epee::net_utils::network_address default_remote; + }; + + boost::optional<std::vector<proxy>> get_proxies(const boost::program_options::variables_map& vm); + boost::optional<std::vector<anonymous_inbound>> get_anonymous_inbounds(const boost::program_options::variables_map& vm); + + //! \return True if `commnd` is filtered (ignored/dropped) for `address` + bool is_filtered_command(epee::net_utils::network_address const& address, int command); + + // hides boost::future and chrono stuff from mondo template file + boost::optional<boost::asio::ip::tcp::socket> + socks_connect_internal(const std::atomic<bool>& stop_signal, boost::asio::io_service& service, const boost::asio::ip::tcp::endpoint& proxy, const epee::net_utils::network_address& remote); + + template<class base_type> struct p2p_connection_context_t: base_type //t_payload_net_handler::connection_context //public net_utils::connection_context_base { @@ -78,53 +126,124 @@ namespace nodetool typedef COMMAND_HANDSHAKE_T<typename t_payload_net_handler::payload_type> COMMAND_HANDSHAKE; typedef COMMAND_TIMED_SYNC_T<typename t_payload_net_handler::payload_type> COMMAND_TIMED_SYNC; + typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context>> net_server; + + struct network_zone; + using connect_func = boost::optional<p2p_connection_context>(network_zone&, epee::net_utils::network_address const&); + + struct config + { + config() + : m_net_config(), + m_peer_id(crypto::rand<uint64_t>()), + m_support_flags(0) + {} + + network_config m_net_config; + uint64_t m_peer_id; + uint32_t m_support_flags; + }; + + struct network_zone + { + network_zone() + : m_connect(nullptr), + m_net_server(epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + network_zone(boost::asio::io_service& public_service) + : m_connect(nullptr), + m_net_server(public_service, epee::net_utils::e_connection_type_P2P), + m_bind_ip(), + m_port(), + m_our_address(), + m_peerlist(), + m_config{}, + m_proxy_address(), + m_current_number_of_out_peers(0), + m_current_number_of_in_peers(0), + m_can_pingback(false) + { + set_config_defaults(); + } + + connect_func* m_connect; + net_server m_net_server; + std::string m_bind_ip; + std::string m_port; + epee::net_utils::network_address m_our_address; // in anonymity networks + peerlist_manager m_peerlist; + config m_config; + boost::asio::ip::tcp::endpoint m_proxy_address; + std::atomic<unsigned int> m_current_number_of_out_peers; + std::atomic<unsigned int> m_current_number_of_in_peers; + bool m_can_pingback; + + private: + void set_config_defaults() noexcept + { + // at this moment we have a hardcoded config + m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; + m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; + m_config.m_net_config.config_id = 0; + m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; + m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; + m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; + m_config.m_support_flags = 0; // only set in public zone + } + }; + public: typedef t_payload_net_handler payload_net_handler; node_server(t_payload_net_handler& payload_handler) - :m_payload_handler(payload_handler), - m_current_number_of_out_peers(0), - m_current_number_of_in_peers(0), - m_allow_local_ip(false), - m_hide_my_port(false), - m_no_igd(false), - m_offline(false), - m_save_graph(false), - is_closing(false), - m_net_server( epee::net_utils::e_connection_type_P2P ) // this is a P2P connection of the main p2p node server, because this is class node_server<> - {} - virtual ~node_server() + : m_payload_handler(payload_handler), + m_external_port(0), + m_allow_local_ip(false), + m_hide_my_port(false), + m_no_igd(false), + m_offline(false), + m_save_graph(false), + is_closing(false), + m_network_id() {} + virtual ~node_server(); static void init_options(boost::program_options::options_description& desc); bool run(); + network_zone& add_zone(epee::net_utils::zone zone); bool init(const boost::program_options::variables_map& vm); bool deinit(); bool send_stop_signal(); uint32_t get_this_peer_port(){return m_listening_port;} t_payload_net_handler& get_payload_object(); - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - a & m_peerlist; - if (ver == 0) - { - // from v1, we do not store the peer id anymore - peerid_type peer_id = AUTO_VAL_INIT (peer_id); - a & peer_id; - } - } // debug functions bool log_peerlist(); bool log_connections(); - virtual uint64_t get_connections_count(); - size_t get_outgoing_connections_count(); - size_t get_incoming_connections_count(); - peerlist_manager& get_peerlist_manager(){return m_peerlist;} - void delete_out_connections(size_t count); - void delete_in_connections(size_t count); + + // These functions only return information for the "public" zone + virtual uint64_t get_public_connections_count(); + size_t get_public_outgoing_connections_count(); + size_t get_public_white_peers_count(); + size_t get_public_gray_peers_count(); + void get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white); + size_t get_zone_count() const { return m_network_zones.size(); } + + void change_max_out_public_peers(size_t count); + void change_max_in_public_peers(size_t count); virtual bool block_host(const epee::net_utils::network_address &adress, time_t seconds = P2P_IP_BLOCKTIME); virtual bool unblock_host(const epee::net_utils::network_address &address); virtual std::map<std::string, time_t> get_blocked_hosts() { CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); return m_blocked_hosts; } @@ -150,6 +269,9 @@ namespace nodetool CHAIN_LEVIN_NOTIFY_MAP2(p2p_connection_context); //move levin_commands_handler interface notify(...) callbacks into nothing BEGIN_INVOKE_MAP2(node_server) + if (is_filtered_command(context.m_remote_address, command)) + return LEVIN_ERROR_CONNECTION_HANDLER_NOT_DEFINED; + HANDLE_INVOKE_T2(COMMAND_HANDSHAKE, &node_server::handle_handshake) HANDLE_INVOKE_T2(COMMAND_TIMED_SYNC, &node_server::handle_timed_sync) HANDLE_INVOKE_T2(COMMAND_PING, &node_server::handle_ping) @@ -178,7 +300,7 @@ namespace nodetool bool make_default_peer_id(); bool make_default_config(); bool store_config(); - bool check_trust(const proof_of_trust& tr); + bool check_trust(const proof_of_trust& tr, epee::net_utils::zone zone_type); //----------------- levin_commands_handler ------------------------------------------------------------- @@ -186,8 +308,7 @@ namespace nodetool virtual void on_connection_close(p2p_connection_context& context); virtual void callback(p2p_connection_context& context); //----------------- i_p2p_endpoint ------------------------------------------------------------- - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections); - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context); + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections); virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context); virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context); virtual bool drop_connection(const epee::net_utils::connection_context_base& context); @@ -204,7 +325,7 @@ namespace nodetool ); bool idle_worker(); bool handle_remote_peerlist(const std::vector<peerlist_entry>& peerlist, time_t local_time, const epee::net_utils::connection_context_base& context); - bool get_local_node_data(basic_node_data& node_data); + bool get_local_node_data(basic_node_data& node_data, const network_zone& zone); //bool get_local_handshake_data(handshake_data& hshd); bool merge_peerlist_with_local(const std::vector<peerlist_entry>& bs); @@ -216,7 +337,7 @@ namespace nodetool bool do_peer_timed_sync(const epee::net_utils::connection_context_base& context, peerid_type peer_id); bool make_new_connection_from_anchor_peerlist(const std::vector<anchor_peerlist_entry>& anchor_peerlist); - bool make_new_connection_from_peerlist(bool use_white_list); + bool make_new_connection_from_peerlist(network_zone& zone, bool use_white_list); bool try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist = false, uint64_t last_seen_stamp = 0, PeerType peer_type = white, uint64_t first_seen_stamp = 0); size_t get_random_index_with_fixed_probability(size_t max_index); bool is_peer_used(const peerlist_entry& peer); @@ -227,7 +348,7 @@ namespace nodetool template<class t_callback> bool try_ping(basic_node_data& node_data, p2p_connection_context& context, const t_callback &cb); bool try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f); - bool make_expected_connections_count(PeerType peer_type, size_t expected_connections); + bool make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections); void cache_connect_fail_info(const epee::net_utils::network_address& addr); bool is_addr_recently_failed(const epee::net_utils::network_address& addr); bool is_priority_node(const epee::net_utils::network_address& na); @@ -240,14 +361,8 @@ namespace nodetool template <class Container> bool parse_peers_and_add_to_container(const boost::program_options::variables_map& vm, const command_line::arg_descriptor<std::vector<std::string> > & arg, Container& container); - bool set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_out_peers() const { return m_config.m_net_config.max_out_connection_count; } - bool get_current_out_peers() const { return m_current_number_of_out_peers; } - - bool set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max); - bool get_max_in_peers() const { return m_config.m_net_config.max_in_connection_count; } - bool get_current_in_peers() const { return m_current_number_of_in_peers; } - + bool set_max_out_peers(network_zone& zone, int64_t max); + bool set_max_in_peers(network_zone& zone, int64_t max); bool set_tos_flag(const boost::program_options::variables_map& vm, int limit); bool set_rate_up_limit(const boost::program_options::variables_map& vm, int64_t limit); @@ -255,6 +370,11 @@ namespace nodetool bool set_rate_limit(const boost::program_options::variables_map& vm, int64_t limit); bool has_too_many_connections(const epee::net_utils::network_address &address); + uint64_t get_connections_count(); + size_t get_incoming_connections_count(); + size_t get_incoming_connections_count(network_zone&); + size_t get_outgoing_connections_count(); + size_t get_outgoing_connections_count(network_zone&); bool check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp); bool gray_peerlist_housekeeping(); @@ -272,25 +392,7 @@ namespace nodetool std::string print_connections_container(); - typedef epee::net_utils::boosted_tcp_server<epee::levin::async_protocol_handler<p2p_connection_context> > net_server; - - struct config - { - network_config m_net_config; - uint64_t m_peer_id; - uint32_t m_support_flags; - - BEGIN_KV_SERIALIZE_MAP() - KV_SERIALIZE(m_net_config) - KV_SERIALIZE(m_peer_id) - KV_SERIALIZE(m_support_flags) - END_KV_SERIALIZE_MAP() - }; - public: - config m_config; // TODO was private, add getters? - std::atomic<unsigned int> m_current_number_of_out_peers; - std::atomic<unsigned int> m_current_number_of_in_peers; void set_save_graph(bool save_graph) { @@ -304,7 +406,6 @@ namespace nodetool bool m_first_connection_maker_call; uint32_t m_listening_port; uint32_t m_external_port; - uint32_t m_ip_address; bool m_allow_local_ip; bool m_hide_my_port; bool m_no_igd; @@ -316,7 +417,7 @@ namespace nodetool //connections_indexed_container m_connections; t_payload_net_handler& m_payload_handler; - peerlist_manager m_peerlist; + peerlist_storage m_peerlist_storage; epee::math_helper::once_a_time_seconds<P2P_DEFAULT_HANDSHAKE_INTERVAL> m_peer_handshake_idle_maker_interval; epee::math_helper::once_a_time_seconds<1> m_connections_maker_interval; @@ -324,8 +425,6 @@ namespace nodetool epee::math_helper::once_a_time_seconds<60> m_gray_peerlist_housekeeping_interval; epee::math_helper::once_a_time_seconds<3600, false> m_incoming_connections_interval; - std::string m_bind_ip; - std::string m_port; #ifdef ALLOW_DEBUG_COMMANDS uint64_t m_last_stat_request_time; #endif @@ -333,11 +432,22 @@ namespace nodetool std::vector<epee::net_utils::network_address> m_exclusive_peers; std::vector<epee::net_utils::network_address> m_seed_nodes; bool m_fallback_seed_nodes_added; - std::list<nodetool::peerlist_entry> m_command_line_peers; + std::vector<nodetool::peerlist_entry> m_command_line_peers; uint64_t m_peer_livetime; //keep connections to initiate some interactions - net_server m_net_server; - boost::uuids::uuid m_network_id; + + + static boost::optional<p2p_connection_context> public_connect(network_zone&, epee::net_utils::network_address const&); + static boost::optional<p2p_connection_context> socks_connect(network_zone&, epee::net_utils::network_address const&); + + + /* A `std::map` provides constant iterators and key/value pointers even with + inserts/erases to _other_ elements. This makes the configuration step easier + since references can safely be stored on the stack. Do not insert/erase + after configuration and before destruction, lock safety would need to be + added. `std::map::operator[]` WILL insert! */ + std::map<epee::net_utils::zone, network_zone> m_network_zones; + std::map<epee::net_utils::network_address, time_t> m_conn_fails_cache; epee::critical_section m_conn_fails_cache_lock; @@ -351,6 +461,7 @@ namespace nodetool boost::mutex m_used_stripe_peers_mutex; std::array<std::list<epee::net_utils::network_address>, 1 << CRYPTONOTE_PRUNING_LOG_STRIPES> m_used_stripe_peers; + boost::uuids::uuid m_network_id; cryptonote::network_type m_nettype; }; @@ -364,6 +475,8 @@ namespace nodetool extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_priority_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_add_exclusive_node; extern const command_line::arg_descriptor<std::vector<std::string> > arg_p2p_seed_node; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_proxy; + extern const command_line::arg_descriptor<std::vector<std::string> > arg_anonymous_inbound; extern const command_line::arg_descriptor<bool> arg_p2p_hide_my_port; extern const command_line::arg_descriptor<bool> arg_no_igd; @@ -380,3 +493,4 @@ namespace nodetool } POP_WARNINGS + diff --git a/src/p2p/net_node.inl b/src/p2p/net_node.inl index c594984d4..471fdda0d 100644 --- a/src/p2p/net_node.inl +++ b/src/p2p/net_node.inl @@ -31,25 +31,34 @@ // IP blocking adapted from Boolberry #include <algorithm> +#include <boost/bind.hpp> #include <boost/date_time/posix_time/posix_time.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/optional/optional.hpp> #include <boost/thread/thread.hpp> #include <boost/uuid/uuid_io.hpp> -#include <boost/bind.hpp> #include <atomic> +#include <functional> +#include <limits> +#include <memory> +#include <tuple> +#include <vector> #include "version.h" #include "string_tools.h" #include "common/util.h" #include "common/dns_utils.h" #include "common/pruning.h" +#include "net/error.h" #include "net/net_helper.h" #include "math_helper.h" +#include "misc_log_ex.h" #include "p2p_protocol_defs.h" -#include "net_peerlist_boost_serialization.h" #include "net/local_ip.h" #include "crypto/crypto.h" #include "storages/levin_abstract_invoke2.h" #include "cryptonote_core/cryptonote_core.h" +#include "net/parse.h" #include <miniupnp/miniupnpc/miniupnpc.h> #include <miniupnp/miniupnpc/upnpcommands.h> @@ -64,6 +73,20 @@ namespace nodetool { + template<class t_payload_net_handler> + node_server<t_payload_net_handler>::~node_server() + { + // tcp server uses io_service in destructor, and every zone uses + // io_service from public zone. + for (auto current = m_network_zones.begin(); current != m_network_zones.end(); /* below */) + { + if (current->first != epee::net_utils::zone::public_) + current = m_network_zones.erase(current); + else + ++current; + } + } + //----------------------------------------------------------------------------------- inline bool append_net_address(std::vector<epee::net_utils::network_address> & seed_nodes, std::string const & addr, uint16_t default_port); //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -77,6 +100,8 @@ namespace nodetool command_line::add_arg(desc, arg_p2p_add_priority_node); command_line::add_arg(desc, arg_p2p_add_exclusive_node); command_line::add_arg(desc, arg_p2p_seed_node); + command_line::add_arg(desc, arg_proxy); + command_line::add_arg(desc, arg_anonymous_inbound); command_line::add_arg(desc, arg_p2p_hide_my_port); command_line::add_arg(desc, arg_no_igd); command_line::add_arg(desc, arg_out_peers); @@ -91,84 +116,14 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init_config() { - // TRY_ENTRY(); - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ifstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - // first try reading in portable mode - boost::archive::portable_binary_iarchive a(p2p_data); - a >> *this; - } - catch (...) - { - // if failed, try reading in unportable mode - boost::filesystem::copy_file(state_file_path, state_file_path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); - p2p_data.close(); - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::in); - if(!p2p_data.fail()) - { - try - { - boost::archive::binary_iarchive a(p2p_data); - a >> *this; - } - catch (const std::exception &e) - { - MWARNING("Failed to load p2p config file, falling back to default config"); - m_peerlist = peerlist_manager(); // it was probably half clobbered by the failed load - make_default_config(); - } - } - else - { - make_default_config(); - } - } - }else - { - make_default_config(); - } - -#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED - std::list<peerlist_entry> plw; - while (m_peerlist.get_white_peers_count()) - { - plw.push_back(peerlist_entry()); - m_peerlist.get_white_peer_by_index(plw.back(), 0); - m_peerlist.remove_from_peer_white(plw.back()); - } - for (auto &e:plw) - m_peerlist.append_with_peer_white(e); - - std::list<peerlist_entry> plg; - while (m_peerlist.get_gray_peers_count()) - { - plg.push_back(peerlist_entry()); - m_peerlist.get_gray_peer_by_index(plg.back(), 0); - m_peerlist.remove_from_peer_gray(plg.back()); - } - for (auto &e:plg) - m_peerlist.append_with_peer_gray(e); -#endif - - // always recreate a new peer id - make_default_peer_id(); - - //at this moment we have hardcoded config - m_config.m_net_config.handshake_interval = P2P_DEFAULT_HANDSHAKE_INTERVAL; - m_config.m_net_config.packet_max_size = P2P_DEFAULT_PACKET_MAX_SIZE; //20 MB limit - m_config.m_net_config.config_id = 0; // initial config - m_config.m_net_config.connection_timeout = P2P_DEFAULT_CONNECTION_TIMEOUT; - m_config.m_net_config.ping_connection_timeout = P2P_DEFAULT_PING_CONNECTION_TIMEOUT; - m_config.m_net_config.send_peerlist_sz = P2P_DEFAULT_PEERS_IN_HANDSHAKE; - m_config.m_support_flags = P2P_SUPPORT_FLAGS; + auto storage = peerlist_storage::open(m_config_folder + "/" + P2P_NET_DATA_FILENAME); + if (storage) + m_peerlist_storage = std::move(*storage); + m_network_zones[epee::net_utils::zone::public_].m_config.m_support_flags = P2P_SUPPORT_FLAGS; m_first_connection_maker_call = true; + CATCH_ENTRY_L0("node_server::init_config", false); return true; } @@ -176,17 +131,26 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::for_each_connection(std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + } } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::for_connection(const boost::uuids::uuid &connection_id, std::function<bool(typename t_payload_net_handler::connection_context&, peerid_type, uint32_t)> f) { - return m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ - return f(cntx, cntx.peer_id, cntx.support_flags); - }); + for(auto& zone : m_network_zones) + { + const bool result = zone.second.m_net_server.get_config_object().for_connection(connection_id, [&](p2p_connection_context& cntx){ + return f(cntx, cntx.peer_id, cntx.support_flags); + }); + if (result) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> @@ -206,36 +170,33 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_peer_id() - { - m_config.m_peer_id = crypto::rand<uint64_t>(); - return true; - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_default_config() - { - return make_default_peer_id(); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::block_host(const epee::net_utils::network_address &addr, time_t seconds) { + if(!addr.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_blocked_hosts_lock); m_blocked_hosts[addr.host_str()] = time(nullptr) + seconds; - // drop any connection to that IP - std::list<boost::uuids::uuid> conns; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + // drop any connection to that address. This should only have to look into + // the zone related to the connection, but really make sure everything is + // swept ... + std::vector<boost::uuids::uuid> conns; + for(auto& zone : m_network_zones) { - if (cntxt.m_remote_address.is_same_host(addr)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - conns.push_back(cntxt.m_connection_id); - } - return true; - }); - for (const auto &c: conns) - m_net_server.get_config_object().close(c); + if (cntxt.m_remote_address.is_same_host(addr)) + { + conns.push_back(cntxt.m_connection_id); + } + return true; + }); + for (const auto &c: conns) + zone.second.m_net_server.get_config_object().close(c); + + conns.clear(); + } MCLOG_CYAN(el::Level::Info, "global", "Host " << addr.host_str() << " blocked."); return true; @@ -256,6 +217,9 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::add_host_fail(const epee::net_utils::network_address &address) { + if(!address.is_blockable()) + return false; + CRITICAL_REGION_LOCAL(m_host_fails_score_lock); uint64_t fails = ++m_host_fails_score[address.host_str()]; MDEBUG("Host " << address.host_str() << " fail score=" << fails); @@ -270,12 +234,6 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::parse_peer_from_string(epee::net_utils::network_address& pe, const std::string& node_addr, uint16_t default_port) - { - return epee::net_utils::create_network_address(pe, node_addr, default_port); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::handle_command_line( const boost::program_options::variables_map& vm ) @@ -284,8 +242,11 @@ namespace nodetool bool stagenet = command_line::get_arg(vm, cryptonote::arg_stagenet_on); m_nettype = testnet ? cryptonote::TESTNET : stagenet ? cryptonote::STAGENET : cryptonote::MAINNET; - m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); - m_port = command_line::get_arg(vm, arg_p2p_bind_port); + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + public_zone.m_connect = &public_connect; + public_zone.m_bind_ip = command_line::get_arg(vm, arg_p2p_bind_ip); + public_zone.m_port = command_line::get_arg(vm, arg_p2p_bind_port); + public_zone.m_can_pingback = true; m_external_port = command_line::get_arg(vm, arg_p2p_external_port); m_allow_local_ip = command_line::get_arg(vm, arg_p2p_allow_local_ip); m_no_igd = command_line::get_arg(vm, arg_no_igd); @@ -299,14 +260,20 @@ namespace nodetool nodetool::peerlist_entry pe = AUTO_VAL_INIT(pe); pe.id = crypto::rand<uint64_t>(); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(pe.adr, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - m_command_line_peers.push_back(pe); + add_zone(adr->get_zone()); + pe.adr = std::move(*adr); + m_command_line_peers.push_back(std::move(pe)); continue; } + CHECK_AND_ASSERT_MES( + adr == net::error::unsupported_address, false, "Bad address (\"" << pr_str << "\"): " << adr.error().message() + ); + std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -343,10 +310,13 @@ namespace nodetool if(command_line::has_arg(vm, arg_p2p_hide_my_port)) m_hide_my_port = true; - if ( !set_max_out_peers(vm, command_line::get_arg(vm, arg_out_peers) ) ) + if ( !set_max_out_peers(public_zone, command_line::get_arg(vm, arg_out_peers) ) ) return false; + else + m_payload_handler.set_max_out_peers(public_zone.m_config.m_net_config.max_out_connection_count); + - if ( !set_max_in_peers(vm, command_line::get_arg(vm, arg_in_peers) ) ) + if ( !set_max_in_peers(public_zone, command_line::get_arg(vm, arg_in_peers) ) ) return false; if ( !set_tos_flag(vm, command_line::get_arg(vm, arg_tos_flag) ) ) @@ -361,6 +331,58 @@ namespace nodetool if ( !set_rate_limit(vm, command_line::get_arg(vm, arg_limit_rate) ) ) return false; + + auto proxies = get_proxies(vm); + if (!proxies) + return false; + + for (auto& proxy : *proxies) + { + network_zone& zone = add_zone(proxy.zone); + if (zone.m_connect != nullptr) + { + MERROR("Listed --" << arg_proxy.name << " twice with " << epee::net_utils::zone_to_string(proxy.zone)); + return false; + } + zone.m_connect = &socks_connect; + zone.m_proxy_address = std::move(proxy.address); + + if (!set_max_out_peers(zone, proxy.max_connections)) + return false; + } + + for (const auto& zone : m_network_zones) + { + if (zone.second.m_connect == nullptr) + { + MERROR("Set outgoing peer for " << epee::net_utils::zone_to_string(zone.first) << " but did not set --" << arg_proxy.name); + return false; + } + } + + auto inbounds = get_anonymous_inbounds(vm); + if (!inbounds) + return false; + + for (auto& inbound : *inbounds) + { + network_zone& zone = add_zone(inbound.our_address.get_zone()); + + if (!zone.m_bind_ip.empty()) + { + MERROR("Listed --" << arg_anonymous_inbound.name << " twice with " << epee::net_utils::zone_to_string(inbound.our_address.get_zone()) << " network"); + return false; + } + + zone.m_bind_ip = std::move(inbound.local_ip); + zone.m_port = std::move(inbound.local_port); + zone.m_net_server.set_default_remote(std::move(inbound.default_remote)); + zone.m_our_address = std::move(inbound.our_address); + + if (!set_max_in_peers(zone, inbound.max_connections)) + return false; + } + return true; } //----------------------------------------------------------------------------------- @@ -442,7 +464,17 @@ namespace nodetool } return full_addrs; } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + typename node_server<t_payload_net_handler>::network_zone& node_server<t_payload_net_handler>::add_zone(const epee::net_utils::zone zone) + { + const auto zone_ = m_network_zones.lower_bound(zone); + if (zone_ != m_network_zones.end() && zone_->first == zone) + return zone_->second; + network_zone& public_zone = m_network_zones[epee::net_utils::zone::public_]; + return m_network_zones.emplace_hint(zone_, std::piecewise_construct, std::make_tuple(zone), std::tie(public_zone.m_net_server.get_io_service()))->second; + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::init(const boost::program_options::variables_map& vm) @@ -559,45 +591,81 @@ namespace nodetool MDEBUG("Number of seed nodes: " << m_seed_nodes.size()); m_config_folder = command_line::get_arg(vm, cryptonote::arg_data_dir); + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); - if ((m_nettype == cryptonote::MAINNET && m_port != std::to_string(::config::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::TESTNET && m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) - || (m_nettype == cryptonote::STAGENET && m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { - m_config_folder = m_config_folder + "/" + m_port; + if ((m_nettype == cryptonote::MAINNET && public_zone.m_port != std::to_string(::config::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::TESTNET && public_zone.m_port != std::to_string(::config::testnet::P2P_DEFAULT_PORT)) + || (m_nettype == cryptonote::STAGENET && public_zone.m_port != std::to_string(::config::stagenet::P2P_DEFAULT_PORT))) { + m_config_folder = m_config_folder + "/" + public_zone.m_port; } res = init_config(); CHECK_AND_ASSERT_MES(res, false, "Failed to init config."); - res = m_peerlist.init(m_allow_local_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + for (auto& zone : m_network_zones) + { + res = zone.second.m_peerlist.init(m_peerlist_storage.take_zone(zone.first), m_allow_local_ip); + CHECK_AND_ASSERT_MES(res, false, "Failed to init peerlist."); + } + for(const auto& p: m_command_line_peers) + m_network_zones.at(p.adr.get_zone()).m_peerlist.append_with_peer_white(p); - for(auto& p: m_command_line_peers) - m_peerlist.append_with_peer_white(p); +// all peers are now setup +#ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED + for (auto& zone : m_network_zones) + { + std::list<peerlist_entry> plw; + while (zone.second.m_peerlist.get_white_peers_count()) + { + plw.push_back(peerlist_entry()); + zone.second.m_peerlist.get_white_peer_by_index(plw.back(), 0); + zone.second.m_peerlist.remove_from_peer_white(plw.back()); + } + for (auto &e:plw) + zone.second.m_peerlist.append_with_peer_white(e); + + std::list<peerlist_entry> plg; + while (zone.second.m_peerlist.get_gray_peers_count()) + { + plg.push_back(peerlist_entry()); + zone.second.m_peerlist.get_gray_peer_by_index(plg.back(), 0); + zone.second.m_peerlist.remove_from_peer_gray(plg.back()); + } + for (auto &e:plg) + zone.second.m_peerlist.append_with_peer_gray(e); + } +#endif //only in case if we really sure that we have external visible ip m_have_address = true; - m_ip_address = 0; m_last_stat_request_time = 0; //configure self - m_net_server.set_threads_prefix("P2P"); - m_net_server.get_config_object().set_handler(this); - m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; - m_net_server.set_connection_filter(this); + + public_zone.m_net_server.set_threads_prefix("P2P"); // all zones use these threads/asio::io_service // from here onwards, it's online stuff if (m_offline) return res; //try to bind - MINFO("Binding on " << m_bind_ip << ":" << m_port); - res = m_net_server.init_server(m_port, m_bind_ip); - CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().set_handler(this); + zone.second.m_net_server.get_config_object().m_invoke_timeout = P2P_DEFAULT_INVOKE_TIMEOUT; + + if (!zone.second.m_bind_ip.empty()) + { + zone.second.m_net_server.set_connection_filter(this); + MINFO("Binding on " << zone.second.m_bind_ip << ":" << zone.second.m_port); + res = zone.second.m_net_server.init_server(zone.second.m_port, zone.second.m_bind_ip); + CHECK_AND_ASSERT_MES(res, false, "Failed to bind server"); + } + } - m_listening_port = m_net_server.get_binded_port(); - MLOG_GREEN(el::Level::Info, "Net service bound to " << m_bind_ip << ":" << m_listening_port); + m_listening_port = public_zone.m_net_server.get_binded_port(); + MLOG_GREEN(el::Level::Info, "Net service bound to " << public_zone.m_bind_ip << ":" << m_listening_port); if(m_external_port) MDEBUG("External port defined as " << m_external_port); @@ -621,44 +689,45 @@ namespace nodetool mPeersLoggerThread.reset(new boost::thread([&]() { _note("Thread monitor number of peers - start"); - while (!is_closing && !m_net_server.is_stop_signal_sent()) + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + while (!is_closing && !public_zone.m_net_server.is_stop_signal_sent()) { // main loop of thread //number_of_peers = m_net_server.get_config_object().get_connections_count(); - unsigned int number_of_in_peers = 0; - unsigned int number_of_out_peers = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - if (cntxt.m_is_income) - { - ++number_of_in_peers; - } - else + unsigned int number_of_in_peers = 0; + unsigned int number_of_out_peers = 0; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - ++number_of_out_peers; - } - return true; - }); // lambda - - m_current_number_of_in_peers = number_of_in_peers; - m_current_number_of_out_peers = number_of_out_peers; - + if (cntxt.m_is_income) + { + ++number_of_in_peers; + } + else + { + ++number_of_out_peers; + } + return true; + }); // lambda + zone.second.m_current_number_of_in_peers = number_of_in_peers; + zone.second.m_current_number_of_out_peers = number_of_out_peers; + } boost::this_thread::sleep_for(boost::chrono::seconds(1)); } // main loop of thread _note("Thread monitor number of peers - done"); })); // lambda + network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); + public_zone.m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); + public_zone.m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); + //here you can set worker threads count int thrds_count = 10; - - m_net_server.add_idle_handler(boost::bind(&node_server<t_payload_net_handler>::idle_worker, this), 1000); - m_net_server.add_idle_handler(boost::bind(&t_payload_net_handler::on_idle, &m_payload_handler), 1000); - boost::thread::attributes attrs; attrs.set_stack_size(THREAD_STACK_SIZE); - //go to loop MINFO("Run net_service loop( " << thrds_count << " threads)..."); - if(!m_net_server.run_server(thrds_count, true, attrs)) + if(!public_zone.m_net_server.run_server(thrds_count, true, attrs)) { LOG_ERROR("Failed to run net tcp server!"); } @@ -666,23 +735,34 @@ namespace nodetool MINFO("net_service loop stopped."); return true; } - + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + uint64_t node_server<t_payload_net_handler>::get_public_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_net_server.get_config_object().get_connections_count(); + } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> uint64_t node_server<t_payload_net_handler>::get_connections_count() { - return m_net_server.get_config_object().get_connections_count(); + std::uint64_t count = 0; + for (auto& zone : m_network_zones) + count += zone.second.m_net_server.get_config_object().get_connections_count(); + return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::deinit() { kill(); - m_peerlist.deinit(); if (!m_offline) { - m_net_server.deinit_server(); + for(auto& zone : m_network_zones) + zone.second.m_net_server.deinit_server(); // remove UPnP port mapping if(!m_no_igd) delete_upnp_port_mapping(m_listening_port); @@ -693,28 +773,25 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::store_config() { - TRY_ENTRY(); + if (!tools::create_directories_if_necessary(m_config_folder)) { - MWARNING("Failed to create data directory: " << m_config_folder); + MWARNING("Failed to create data directory \"" << m_config_folder); return false; } - std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; - std::ofstream p2p_data; - p2p_data.open( state_file_path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); - if(p2p_data.fail()) + peerlist_types active{}; + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(active); + + const std::string state_file_path = m_config_folder + "/" + P2P_NET_DATA_FILENAME; + if (!m_peerlist_storage.store(state_file_path, active)) { MWARNING("Failed to save config to file " << state_file_path); return false; - }; - - boost::archive::portable_binary_oarchive a(p2p_data); - a << *this; - return true; - CATCH_ENTRY_L0("blockchain_storage::save", false); - + } + CATCH_ENTRY_L0("node_server::store", false); return true; } //----------------------------------------------------------------------------------- @@ -722,36 +799,38 @@ namespace nodetool bool node_server<t_payload_net_handler>::send_stop_signal() { MDEBUG("[node] sending stop signal"); - m_net_server.send_stop_signal(); + for (auto& zone : m_network_zones) + zone.second.m_net_server.send_stop_signal(); MDEBUG("[node] Stop signal sent"); - std::list<boost::uuids::uuid> connection_ids; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - connection_ids.push_back(cntxt.m_connection_id); - return true; - }); - for (const auto &connection_id: connection_ids) - m_net_server.get_config_object().close(connection_id); - + for (auto& zone : m_network_zones) + { + std::list<boost::uuids::uuid> connection_ids; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { + connection_ids.push_back(cntxt.m_connection_id); + return true; + }); + for (const auto &connection_id: connection_ids) + zone.second.m_net_server.get_config_object().close(connection_id); + } m_payload_handler.stop(); - return true; } //----------------------------------------------------------------------------------- - - template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::do_handshake_with_peer(peerid_type& pi, p2p_connection_context& context_, bool just_take_peerlist) { + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + typename COMMAND_HANDSHAKE::request arg; typename COMMAND_HANDSHAKE::response rsp; - get_local_node_data(arg.node_data); + get_local_node_data(arg.node_data, zone); m_payload_handler.get_payload_sync_data(arg.payload_data); epee::simple_event ev; std::atomic<bool> hsh_result(false); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, m_net_server.get_config_object(), + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_HANDSHAKE::response>(context_.m_connection_id, COMMAND_HANDSHAKE::ID, arg, zone.m_net_server.get_config_object(), [this, &pi, &ev, &hsh_result, &just_take_peerlist, &context_](int code, const typename COMMAND_HANDSHAKE::response& rsp, p2p_connection_context& context) { epee::misc_utils::auto_scope_leave_caller scope_exit_handler = epee::misc_utils::create_scope_leave_handler([&](){ev.raise();}); @@ -785,13 +864,17 @@ namespace nodetool } pi = context.peer_id = rsp.node_data.peer_id; - m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(rsp.node_data.peer_id, context.m_remote_address, context.m_pruning_seed); - if(rsp.node_data.peer_id == m_config.m_peer_id) + // move + for (auto const& zone : m_network_zones) { - LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); - hsh_result = false; - return; + if(rsp.node_data.peer_id == zone.second.m_config.m_peer_id) + { + LOG_DEBUG_CC(context, "Connection to self detected, dropping connection"); + hsh_result = false; + return; + } } LOG_INFO_CC(context, "New connection handshaked, pruning seed " << epee::string_tools::to_string_hex(context.m_pruning_seed)); LOG_DEBUG_CC(context, " COMMAND_HANDSHAKE INVOKED OK"); @@ -810,7 +893,7 @@ namespace nodetool if(!hsh_result) { LOG_WARNING_CC(context_, "COMMAND_HANDSHAKE Failed"); - m_net_server.get_config_object().close(context_.m_connection_id); + m_network_zones.at(context_.m_remote_address.get_zone()).m_net_server.get_config_object().close(context_.m_connection_id); } else { @@ -829,7 +912,8 @@ namespace nodetool typename COMMAND_TIMED_SYNC::request arg = AUTO_VAL_INIT(arg); m_payload_handler.get_payload_sync_data(arg.payload_data); - bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(context_.m_remote_address.get_zone()); + bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_TIMED_SYNC::response>(context_.m_connection_id, COMMAND_TIMED_SYNC::ID, arg, zone.m_net_server.get_config_object(), [this](int code, const typename COMMAND_TIMED_SYNC::response& rsp, p2p_connection_context& context) { context.m_in_timedsync = false; @@ -842,11 +926,11 @@ namespace nodetool if(!handle_remote_peerlist(rsp.local_peerlist_new, rsp.local_time, context)) { LOG_WARNING_CC(context, "COMMAND_TIMED_SYNC: failed to handle_remote_peerlist(...), closing connection."); - m_net_server.get_config_object().close(context.m_connection_id ); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id ); add_host_fail(context.m_remote_address); } if(!context.m_is_income) - m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); + m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.set_peer_just_seen(context.peer_id, context.m_remote_address, context.m_pruning_seed); m_payload_handler.process_payload_sync_data(rsp.payload_data, context, false); }); @@ -874,53 +958,61 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const peerlist_entry& peer) { - - if(m_config.m_peer_id == peer.id) - return true;//dont make connections to ourself + for(const auto& zone : m_network_zones) + if(zone.second.m_config.m_peer_id == peer.id) + return true;//dont make connections to ourself bool used = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - used = true; - return false;//stop enumerating - } - return true; - }); + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); - return used; + if(used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_peer_used(const anchor_peerlist_entry& peer) { - if(m_config.m_peer_id == peer.id) { - return true;//dont make connections to ourself - } - - bool used = false; - - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) - { - used = true; - - return false;//stop enumerating + for(auto& zone : m_network_zones) { + if(zone.second.m_config.m_peer_id == peer.id) { + return true;//dont make connections to ourself } - - return true; - }); - - return used; + bool used = false; + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.peer_id == peer.id || (!cntxt.m_is_income && peer.adr == cntxt.m_remote_address)) + { + used = true; + return false;//stop enumerating + } + return true; + }); + if (used) + return true; + } + return false; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::is_addr_connected(const epee::net_utils::network_address& peer) { + const auto zone = m_network_zones.find(peer.get_zone()); + if (zone == m_network_zones.end()) + return false; + bool connected = false; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone->second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if(!cntxt.m_is_income && peer == cntxt.m_remote_address) { @@ -945,47 +1037,44 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_to_connect_and_handshake_with_new_peer(const epee::net_utils::network_address& na, bool just_take_peerlist, uint64_t last_seen_stamp, PeerType peer_type, uint64_t first_seen_stamp) { - if (m_current_number_of_out_peers == m_config.m_net_config.max_out_connection_count) // out peers limit + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) // outgoing connections in zone not possible + return false; + + if (zone.m_current_number_of_out_peers == zone.m_config.m_net_config.max_out_connection_count) // out peers limit { return false; } - else if (m_current_number_of_out_peers > m_config.m_net_config.max_out_connection_count) + else if (zone.m_current_number_of_out_peers > zone.m_config.m_net_config.max_out_connection_count) { - m_net_server.get_config_object().del_out_connections(1); - m_current_number_of_out_peers --; // atomic variable, update time = 1s + zone.m_net_server.get_config_object().del_out_connections(1); + --(zone.m_current_number_of_out_peers); // atomic variable, update time = 1s return false; } + + MDEBUG("Connecting to " << na.str() << "(peer_type=" << peer_type << ", last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = peer_type == anchor; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if(!res) + auto con = zone.m_connect(zone, na); + if(!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str() + LOG_PRINT_CC_PRIORITY_NODE(is_priority, bool(con), "Connect failed to " << na.str() /*<< ", try " << try_count*/); //m_peerlist.set_peer_unreachable(pe); return false; } + con->m_anchor = peer_type == anchor; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, just_take_peerlist); + bool res = do_handshake_with_peer(pi, *con, just_take_peerlist); if(!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str() /*<< ", try " << try_count*/); return false; @@ -993,8 +1082,8 @@ namespace nodetool if(just_take_peerlist) { - m_net_server.get_config_object().close(con.m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + zone.m_net_server.get_config_object().close(con->m_connection_id); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1004,8 +1093,8 @@ namespace nodetool time_t last_seen; time(&last_seen); pe_local.last_seen = static_cast<int64_t>(last_seen); - pe_local.pruning_seed = con.m_pruning_seed; - m_peerlist.append_with_peer_white(pe_local); + pe_local.pruning_seed = con->m_pruning_seed; + zone.m_peerlist.append_with_peer_white(pe_local); //update last seen and push it to peerlist manager anchor_peerlist_entry ape = AUTO_VAL_INIT(ape); @@ -1013,52 +1102,46 @@ namespace nodetool ape.id = pi; ape.first_seen = first_seen_stamp ? first_seen_stamp : time(nullptr); - m_peerlist.append_with_peer_anchor(ape); + zone.m_peerlist.append_with_peer_anchor(ape); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK."); return true; } template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::check_connection_and_handshake_with_peer(const epee::net_utils::network_address& na, uint64_t last_seen_stamp) { + network_zone& zone = m_network_zones.at(na.get_zone()); + if (zone.m_connect == nullptr) + return false; + LOG_PRINT_L1("Connecting to " << na.str() << "(last_seen: " << (last_seen_stamp ? epee::misc_utils::get_time_interval_string(time(NULL) - last_seen_stamp):"never") << ")..."); - CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, - "Only IPv4 addresses are supported here"); - const epee::net_utils::ipv4_network_address &ipv4 = na.as<epee::net_utils::ipv4_network_address>(); - - typename net_server::t_connection_context con = AUTO_VAL_INIT(con); - con.m_anchor = false; - bool res = m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), - epee::string_tools::num_to_string_fast(ipv4.port()), - m_config.m_net_config.connection_timeout, - con); - - if (!res) { + auto con = zone.m_connect(zone, na); + if (!con) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Connect failed to " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, p2p_connection_context{}, "Connect failed to " << na.str()); return false; } + con->m_anchor = false; peerid_type pi = AUTO_VAL_INIT(pi); - res = do_handshake_with_peer(pi, con, true); - + const bool res = do_handshake_with_peer(pi, *con, true); if (!res) { bool is_priority = is_priority_node(na); - LOG_PRINT_CC_PRIORITY_NODE(is_priority, con, "Failed to HANDSHAKE with peer " << na.str()); + LOG_PRINT_CC_PRIORITY_NODE(is_priority, *con, "Failed to HANDSHAKE with peer " << na.str()); return false; } - m_net_server.get_config_object().close(con.m_connection_id); + zone.m_net_server.get_config_object().close(con->m_connection_id); - LOG_DEBUG_CC(con, "CONNECTION HANDSHAKED OK AND CLOSED."); + LOG_DEBUG_CC(*con, "CONNECTION HANDSHAKED OK AND CLOSED."); return true; } @@ -1115,7 +1198,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(bool use_white_list) + bool node_server<t_payload_net_handler>::make_new_connection_from_peerlist(network_zone& zone, bool use_white_list) { size_t max_random_index = 0; @@ -1123,7 +1206,7 @@ namespace nodetool size_t try_count = 0; size_t rand_count = 0; - while(rand_count < (max_random_index+1)*3 && try_count < 10 && !m_net_server.is_stop_signal_sent()) + while(rand_count < (max_random_index+1)*3 && try_count < 10 && !zone.m_net_server.is_stop_signal_sent()) { ++rand_count; size_t random_index; @@ -1132,7 +1215,7 @@ namespace nodetool std::deque<size_t> filtered; const size_t limit = use_white_list ? 20 : std::numeric_limits<size_t>::max(); size_t idx = 0; - m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ + zone.m_peerlist.foreach (use_white_list, [&filtered, &idx, limit, next_needed_pruning_stripe](const peerlist_entry &pe){ if (filtered.size() >= limit) return false; if (next_needed_pruning_stripe == 0 || pe.pruning_seed == 0) @@ -1159,7 +1242,7 @@ namespace nodetool for (size_t i = 0; i < filtered.size(); ++i) { peerlist_entry pe; - if (m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) + if (zone.m_peerlist.get_white_peer_by_index(pe, filtered[i]) && pe.adr == na) { MDEBUG("Reusing stripe " << next_needed_pruning_stripe << " peer " << pe.adr.str()); random_index = i; @@ -1173,7 +1256,7 @@ namespace nodetool CHECK_AND_ASSERT_MES(random_index < filtered.size(), false, "random_index < filtered.size() failed!!"); random_index = filtered[random_index]; - CHECK_AND_ASSERT_MES(random_index < (use_white_list ? m_peerlist.get_white_peers_count() : m_peerlist.get_gray_peers_count()), + CHECK_AND_ASSERT_MES(random_index < (use_white_list ? zone.m_peerlist.get_white_peers_count() : zone.m_peerlist.get_gray_peers_count()), false, "random_index < peers size failed!!"); if(tried_peers.count(random_index)) @@ -1181,7 +1264,7 @@ namespace nodetool tried_peers.insert(random_index); peerlist_entry pe = AUTO_VAL_INIT(pe); - bool r = use_white_list ? m_peerlist.get_white_peer_by_index(pe, random_index):m_peerlist.get_gray_peer_by_index(pe, random_index); + bool r = use_white_list ? zone.m_peerlist.get_white_peer_by_index(pe, random_index):zone.m_peerlist.get_gray_peer_by_index(pe, random_index); CHECK_AND_ASSERT_MES(r, false, "Failed to get random peer from peerlist(white:" << use_white_list << ")"); ++try_count; @@ -1224,9 +1307,10 @@ namespace nodetool size_t try_count = 0; size_t current_index = crypto::rand<size_t>()%m_seed_nodes.size(); + const net_server& server = m_network_zones.at(epee::net_utils::zone::public_).m_net_server; while(true) { - if(m_net_server.is_stop_signal_sent()) + if(server.is_stop_signal_sent()) return false; if(try_to_connect_and_handshake_with_new_peer(m_seed_nodes[current_index], true)) @@ -1265,13 +1349,17 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::connections_maker() { + using zone_type = epee::net_utils::zone; + if (m_offline) return true; if (!connect_to_peerlist(m_exclusive_peers)) return false; if (!m_exclusive_peers.empty()) return true; - size_t start_conn_count = get_outgoing_connections_count(); - if(!m_peerlist.get_white_peers_count() && m_seed_nodes.size()) + // Only have seeds in the public zone right now. + + size_t start_conn_count = get_public_outgoing_connections_count(); + if(!get_public_white_peers_count() && m_seed_nodes.size()) { if (!connect_to_seed()) return false; @@ -1279,38 +1367,41 @@ namespace nodetool if (!connect_to_peerlist(m_priority_peers)) return false; - size_t base_expected_white_connections = (m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; - - size_t conn_count = get_outgoing_connections_count(); - while(conn_count < m_config.m_net_config.max_out_connection_count) + for(auto& zone : m_network_zones) { - const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? m_config.m_net_config.max_out_connection_count : base_expected_white_connections; - if(conn_count < expected_white_connections) - { - //start from anchor list - while (get_outgoing_connections_count() < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT - && make_expected_connections_count(anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); - //then do white list - while (get_outgoing_connections_count() < expected_white_connections - && make_expected_connections_count(white, expected_white_connections)); - //then do grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - }else + size_t base_expected_white_connections = (zone.second.m_config.m_net_config.max_out_connection_count*P2P_DEFAULT_WHITELIST_CONNECTIONS_PERCENT)/100; + + size_t conn_count = get_outgoing_connections_count(zone.second); + while(conn_count < zone.second.m_config.m_net_config.max_out_connection_count) { - //start from grey list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(gray, m_config.m_net_config.max_out_connection_count)); - //and then do white list - while (get_outgoing_connections_count() < m_config.m_net_config.max_out_connection_count - && make_expected_connections_count(white, m_config.m_net_config.max_out_connection_count)); + const size_t expected_white_connections = m_payload_handler.get_next_needed_pruning_stripe().second ? zone.second.m_config.m_net_config.max_out_connection_count : base_expected_white_connections; + if(conn_count < expected_white_connections) + { + //start from anchor list + while (get_outgoing_connections_count(zone.second) < P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT + && make_expected_connections_count(zone.second, anchor, P2P_DEFAULT_ANCHOR_CONNECTIONS_COUNT)); + //then do white list + while (get_outgoing_connections_count(zone.second) < expected_white_connections + && make_expected_connections_count(zone.second, white, expected_white_connections)); + //then do grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + }else + { + //start from grey list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, gray, zone.second.m_config.m_net_config.max_out_connection_count)); + //and then do white list + while (get_outgoing_connections_count(zone.second) < zone.second.m_config.m_net_config.max_out_connection_count + && make_expected_connections_count(zone.second, white, zone.second.m_config.m_net_config.max_out_connection_count)); + } + if(zone.second.m_net_server.is_stop_signal_sent()) + return false; + conn_count = get_outgoing_connections_count(zone.second); } - if(m_net_server.is_stop_signal_sent()) - return false; - conn_count = get_outgoing_connections_count(); } - if (start_conn_count == get_outgoing_connections_count() && start_conn_count < m_config.m_net_config.max_out_connection_count) + if (start_conn_count == get_public_outgoing_connections_count() && start_conn_count < m_network_zones.at(zone_type::public_).m_config.m_net_config.max_out_connection_count) { MINFO("Failed to connect to any, trying seeds"); if (!connect_to_seed()) @@ -1321,7 +1412,7 @@ namespace nodetool } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::make_expected_connections_count(PeerType peer_type, size_t expected_connections) + bool node_server<t_payload_net_handler>::make_expected_connections_count(network_zone& zone, PeerType peer_type, size_t expected_connections) { if (m_offline) return false; @@ -1329,14 +1420,14 @@ namespace nodetool std::vector<anchor_peerlist_entry> apl; if (peer_type == anchor) { - m_peerlist.get_and_empty_anchor_peerlist(apl); + zone.m_peerlist.get_and_empty_anchor_peerlist(apl); } - size_t conn_count = get_outgoing_connections_count(); + size_t conn_count = get_outgoing_connections_count(zone); //add new connections from white peers if(conn_count < expected_connections) { - if(m_net_server.is_stop_signal_sent()) + if(zone.m_net_server.is_stop_signal_sent()) return false; MDEBUG("Making expected connection, type " << peer_type << ", " << conn_count << "/" << expected_connections << " connections"); @@ -1345,47 +1436,104 @@ namespace nodetool return false; } - if (peer_type == white && !make_new_connection_from_peerlist(true)) { + if (peer_type == white && !make_new_connection_from_peerlist(zone, true)) { return false; } - if (peer_type == gray && !make_new_connection_from_peerlist(false)) { + if (peer_type == gray && !make_new_connection_from_peerlist(zone, false)) { return false; } } return true; } - //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + size_t node_server<t_payload_net_handler>::get_public_outgoing_connections_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return get_outgoing_connections_count(public_zone->second); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(!cntxt.m_is_income) + if(cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count(network_zone& zone) { size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + zone.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { - if(cntxt.m_is_income) + if(!cntxt.m_is_income) ++count; return true; }); - return count; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_outgoing_connections_count() + { + size_t count = 0; + for(auto& zone : m_network_zones) + count += get_outgoing_connections_count(zone.second); + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_incoming_connections_count() + { + size_t count = 0; + for (auto& zone : m_network_zones) + { + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + if(cntxt.m_is_income) + ++count; + return true; + }); + } + return count; + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_white_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_white_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + size_t node_server<t_payload_net_handler>::get_public_gray_peers_count() + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone == m_network_zones.end()) + return 0; + return public_zone->second.m_peerlist.get_gray_peers_count(); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> + void node_server<t_payload_net_handler>::get_public_peerlist(std::vector<peerlist_entry>& gray, std::vector<peerlist_entry>& white) + { + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + public_zone->second.m_peerlist.get_peerlist(gray, white); + } + //----------------------------------------------------------------------------------- + template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::idle_worker() { m_peer_handshake_idle_maker_interval.do_call(boost::bind(&node_server<t_payload_net_handler>::peer_sync_idle_maker, this)); @@ -1401,9 +1549,11 @@ namespace nodetool { if (m_offline) return true; - if (get_incoming_connections_count() == 0) + + const auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end() && get_incoming_connections_count(public_zone->second) == 0) { - if (m_hide_my_port || m_config.m_net_config.max_in_connection_count == 0) + if (m_hide_my_port || public_zone->second.m_config.m_net_config.max_in_connection_count == 0) { MGINFO("Incoming connections disabled, enable them for full connectivity"); } @@ -1422,15 +1572,18 @@ namespace nodetool MDEBUG("STARTED PEERLIST IDLE HANDSHAKE"); typedef std::list<std::pair<epee::net_utils::connection_context_base, peerid_type> > local_connects_type; local_connects_type cncts; - m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) + for(auto& zone : m_network_zones) { - if(cntxt.peer_id && !cntxt.m_in_timedsync) + zone.second.m_net_server.get_config_object().foreach_connection([&](p2p_connection_context& cntxt) { - cntxt.m_in_timedsync = true; - cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections - } - return true; - }); + if(cntxt.peer_id && !cntxt.m_in_timedsync) + { + cntxt.m_in_timedsync = true; + cncts.push_back(local_connects_type::value_type(cntxt, cntxt.peer_id));//do idle sync only with handshaked connections + } + return true; + }); + } std::for_each(cncts.begin(), cncts.end(), [&](const typename local_connects_type::value_type& vl){do_peer_timed_sync(vl.first, vl.second);}); @@ -1468,19 +1621,30 @@ namespace nodetool std::vector<peerlist_entry> peerlist_ = peerlist; if(!fix_time_delta(peerlist_, local_time, delta)) return false; + + const epee::net_utils::zone zone = context.m_remote_address.get_zone(); + for(const auto& peer : peerlist_) + { + if(peer.adr.get_zone() != zone) + { + MWARNING(context << " sent peerlist from another zone, dropping"); + return false; + } + } + LOG_DEBUG_CC(context, "REMOTE PEERLIST: TIME_DELTA: " << delta << ", remote peerlist size=" << peerlist_.size()); LOG_DEBUG_CC(context, "REMOTE PEERLIST: " << print_peerlist_to_string(peerlist_)); - return m_peerlist.merge_peerlist(peerlist_); + return m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.merge_peerlist(peerlist_); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data) + bool node_server<t_payload_net_handler>::get_local_node_data(basic_node_data& node_data, const network_zone& zone) { time_t local_time; time(&local_time); - node_data.local_time = local_time; - node_data.peer_id = m_config.m_peer_id; - if(!m_hide_my_port) + node_data.local_time = local_time; // \TODO This can be an identifying value across zones (public internet to tor/i2p) ... + node_data.peer_id = zone.m_config.m_peer_id; + if(!m_hide_my_port && zone.m_can_pingback) node_data.my_port = m_external_port ? m_external_port : m_listening_port; else node_data.my_port = 0; @@ -1490,7 +1654,7 @@ namespace nodetool //----------------------------------------------------------------------------------- #ifdef ALLOW_DEBUG_COMMANDS template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr) + bool node_server<t_payload_net_handler>::check_trust(const proof_of_trust& tr, const epee::net_utils::zone zone_type) { uint64_t local_time = time(NULL); uint64_t time_delata = local_time > tr.time ? local_time - tr.time: tr.time - local_time; @@ -1504,9 +1668,11 @@ namespace nodetool MWARNING("check_trust failed to check time conditions, last_stat_request_time=" << m_last_stat_request_time << ", proof_time=" << tr.time); return false; } - if(m_config.m_peer_id != tr.peer_id) + + const network_zone& zone = m_network_zones.at(zone_type); + if(zone.m_config.m_peer_id != tr.peer_id) { - MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << m_config.m_peer_id<< ")"); + MWARNING("check_trust failed: peer_id mismatch (passed " << tr.peer_id << ", expected " << zone.m_config.m_peer_id<< ")"); return false; } crypto::public_key pk = AUTO_VAL_INIT(pk); @@ -1525,12 +1691,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_stat_info(int command, typename COMMAND_REQUEST_STAT_INFO::request& arg, typename COMMAND_REQUEST_STAT_INFO::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - rsp.connections_count = m_net_server.get_config_object().get_connections_count(); + rsp.connections_count = get_connections_count(); rsp.incoming_connections_count = rsp.connections_count - get_outgoing_connections_count(); rsp.version = MONERO_VERSION_FULL; rsp.os_version = tools::get_os_version_string(); @@ -1541,12 +1707,12 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_network_state(int command, COMMAND_REQUEST_NETWORK_STATE::request& arg, COMMAND_REQUEST_NETWORK_STATE::response& rsp, p2p_connection_context& context) { - if(!check_trust(arg.tr)) + if(!check_trust(arg.tr, context.m_remote_address.get_zone())) { drop_connection(context); return 1; } - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { connection_entry ce; ce.adr = cntxt.m_remote_address; @@ -1556,8 +1722,9 @@ namespace nodetool return true; }); - m_peerlist.get_peerlist_full(rsp.local_peerlist_gray, rsp.local_peerlist_white); - rsp.my_id = m_config.m_peer_id; + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + zone.m_peerlist.get_peerlist(rsp.local_peerlist_gray, rsp.local_peerlist_white); + rsp.my_id = zone.m_config.m_peer_id; rsp.local_time = time(NULL); return 1; } @@ -1565,7 +1732,7 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_peer_id(int command, COMMAND_REQUEST_PEER_ID::request& arg, COMMAND_REQUEST_PEER_ID::response& rsp, p2p_connection_context& context) { - rsp.my_id = m_config.m_peer_id; + rsp.my_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } #endif @@ -1573,40 +1740,42 @@ namespace nodetool template<class t_payload_net_handler> int node_server<t_payload_net_handler>::handle_get_support_flags(int command, COMMAND_REQUEST_SUPPORT_FLAGS::request& arg, COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context) { - rsp.support_flags = m_config.m_support_flags; + rsp.support_flags = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_support_flags; return 1; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> void node_server<t_payload_net_handler>::request_callback(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().request_callback(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().request_callback(context.m_connection_id); } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid> &connections) + bool node_server<t_payload_net_handler>::relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { + std::sort(connections.begin(), connections.end()); + auto zone = m_network_zones.begin(); for(const auto& c_id: connections) { - m_net_server.get_config_object().notify(command, data_buff, c_id); + for (;;) + { + if (zone == m_network_zones.end()) + { + MWARNING("Unable to relay all messages, " << epee::net_utils::zone_to_string(c_id.first) << " not available"); + return false; + } + if (c_id.first <= zone->first) + break; + + ++zone; + } + if (zone->first == c_id.first) + zone->second.m_net_server.get_config_object().notify(command, data_buff, c_id.second); } return true; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) - { - std::list<boost::uuids::uuid> connections; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) - { - if(cntxt.peer_id && context.m_connection_id != cntxt.m_connection_id) - connections.push_back(cntxt.m_connection_id); - return true; - }); - return relay_notify_to_list(command, data_buff, connections); - } - //----------------------------------------------------------------------------------- - template<class t_payload_net_handler> void node_server<t_payload_net_handler>::callback(p2p_connection_context& context) { m_payload_handler.on_callback(context); @@ -1615,21 +1784,29 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().notify(command, req_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context) { - int res = m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); + if(is_filtered_command(context.m_remote_address, command)) + return false; + + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + int res = zone.m_net_server.get_config_object().invoke(command, req_buff, resp_buff, context.m_connection_id); return res > 0; } //----------------------------------------------------------------------------------- template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::drop_connection(const epee::net_utils::connection_context_base& context) { - m_net_server.get_config_object().close(context.m_connection_id); + m_network_zones.at(context.m_remote_address.get_zone()).m_net_server.get_config_object().close(context.m_connection_id); return true; } //----------------------------------------------------------------------------------- @@ -1639,18 +1816,21 @@ namespace nodetool if(!node_data.my_port) return false; - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, false, + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), false, "Only IPv4 addresses are supported here"); const epee::net_utils::network_address na = context.m_remote_address; uint32_t actual_ip = na.as<const epee::net_utils::ipv4_network_address>().ip(); - if(!m_peerlist.is_host_allowed(context.m_remote_address)) + network_zone& zone = m_network_zones.at(na.get_zone()); + + if(!zone.m_peerlist.is_host_allowed(context.m_remote_address)) return false; + std::string ip = epee::string_tools::get_ip_string_from_int32(actual_ip); std::string port = epee::string_tools::num_to_string_fast(node_data.my_port); epee::net_utils::network_address address{epee::net_utils::ipv4_network_address(actual_ip, node_data.my_port)}; peerid_type pr = node_data.peer_id; - bool r = m_net_server.connect_async(ip, port, m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( + bool r = zone.m_net_server.connect_async(ip, port, zone.m_config.m_net_config.ping_connection_timeout, [cb, /*context,*/ address, pr, this]( const typename net_server::t_connection_context& ping_context, const boost::system::error_code& ec)->bool { @@ -1670,7 +1850,9 @@ namespace nodetool // GCC 5.1.0 gives error with second use of uint64_t (peerid_type) variable. peerid_type pr_ = pr; - bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, m_net_server.get_config_object(), + network_zone& zone = m_network_zones.at(address.get_zone()); + + bool inv_call_res = epee::net_utils::async_invoke_remote_command2<COMMAND_PING::response>(ping_context.m_connection_id, COMMAND_PING::ID, req, zone.m_net_server.get_config_object(), [=](int code, const COMMAND_PING::response& rsp, p2p_connection_context& context) { if(code <= 0) @@ -1679,20 +1861,21 @@ namespace nodetool return; } + network_zone& zone = m_network_zones.at(address.get_zone()); if(rsp.status != PING_OK_RESPONSE_STATUS_TEXT || pr != rsp.peer_id) { LOG_WARNING_CC(ping_context, "back ping invoke wrong response \"" << rsp.status << "\" from" << address.str() << ", hsh_peer_id=" << pr_ << ", rsp.peer_id=" << rsp.peer_id); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return; } - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); cb(); }); if(!inv_call_res) { LOG_WARNING_CC(ping_context, "back ping invoke failed to " << address.str()); - m_net_server.get_config_object().close(ping_context.m_connection_id); + zone.m_net_server.get_config_object().close(ping_context.m_connection_id); return false; } return true; @@ -1707,13 +1890,16 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::try_get_support_flags(const p2p_connection_context& context, std::function<void(p2p_connection_context&, const uint32_t&)> f) { + if(context.m_remote_address.get_zone() != epee::net_utils::zone::public_) + return false; + COMMAND_REQUEST_SUPPORT_FLAGS::request support_flags_request; bool r = epee::net_utils::async_invoke_remote_command2<typename COMMAND_REQUEST_SUPPORT_FLAGS::response> ( context.m_connection_id, COMMAND_REQUEST_SUPPORT_FLAGS::ID, support_flags_request, - m_net_server.get_config_object(), + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object(), [=](int code, const typename COMMAND_REQUEST_SUPPORT_FLAGS::response& rsp, p2p_connection_context& context_) { if(code < 0) @@ -1742,8 +1928,24 @@ namespace nodetool //fill response rsp.local_time = time(NULL); - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + + const epee::net_utils::zone zone_type = context.m_remote_address.get_zone(); + network_zone& zone = m_network_zones.at(zone_type); + + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); m_payload_handler.get_payload_sync_data(rsp.payload_data); + + /* Tor/I2P nodes receiving connections via forwarding (from tor/i2p daemon) + do not know the address of the connecting peer. This is relayed to them, + iff the node has setup an inbound hidden service. The other peer will have + to use the random peer_id value to link the two. My initial thought is that + the inbound peer should leave the other side marked as `<unknown tor host>`, + etc., because someone could give faulty addresses over Tor/I2P to get the + real peer with that identity banned/blacklisted. */ + + if(!context.m_is_income && zone.m_our_address.get_zone() == zone_type) + rsp.local_peerlist_new.push_back(peerlist_entry{zone.m_our_address, zone.m_config.m_peer_id, std::time(nullptr)}); + LOG_DEBUG_CC(context, "COMMAND_TIMED_SYNC"); return 1; } @@ -1775,7 +1977,9 @@ namespace nodetool return 1; } - if (m_current_number_of_in_peers >= m_config.m_net_config.max_in_connection_count) // in peers limit + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + + if (zone.m_current_number_of_in_peers >= zone.m_config.m_net_config.max_in_connection_count) // in peers limit { LOG_WARNING_CC(context, "COMMAND_HANDSHAKE came, but already have max incoming connections, so dropping this one."); drop_connection(context); @@ -1800,14 +2004,14 @@ namespace nodetool context.peer_id = arg.node_data.peer_id; context.m_in_timedsync = false; - if(arg.node_data.peer_id != m_config.m_peer_id && arg.node_data.my_port) + if(arg.node_data.peer_id != zone.m_config.m_peer_id && arg.node_data.my_port && zone.m_can_pingback) { peerid_type peer_id_l = arg.node_data.peer_id; uint32_t port_l = arg.node_data.my_port; //try ping to be sure that we can add this peer to peer_list try_ping(arg.node_data, context, [peer_id_l, port_l, context, this]() { - CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::ID, void(), + CHECK_AND_ASSERT_MES(context.m_remote_address.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), void(), "Only IPv4 addresses are supported here"); //called only(!) if success pinged, update local peerlist peerlist_entry pe; @@ -1818,7 +2022,7 @@ namespace nodetool pe.last_seen = static_cast<int64_t>(last_seen); pe.id = peer_id_l; pe.pruning_seed = context.m_pruning_seed; - this->m_peerlist.append_with_peer_white(pe); + this->m_network_zones.at(context.m_remote_address.get_zone()).m_peerlist.append_with_peer_white(pe); LOG_DEBUG_CC(context, "PING SUCCESS " << context.m_remote_address.host_str() << ":" << port_l); }); } @@ -1829,8 +2033,8 @@ namespace nodetool }); //fill response - m_peerlist.get_peerlist_head(rsp.local_peerlist_new); - get_local_node_data(rsp.node_data); + zone.m_peerlist.get_peerlist_head(rsp.local_peerlist_new); + get_local_node_data(rsp.node_data, zone); m_payload_handler.get_payload_sync_data(rsp.payload_data); LOG_DEBUG_CC(context, "COMMAND_HANDSHAKE"); return 1; @@ -1841,7 +2045,7 @@ namespace nodetool { LOG_DEBUG_CC(context, "COMMAND_PING"); rsp.status = PING_OK_RESPONSE_STATUS_TEXT; - rsp.peer_id = m_config.m_peer_id; + rsp.peer_id = m_network_zones.at(context.m_remote_address.get_zone()).m_config.m_peer_id; return 1; } //----------------------------------------------------------------------------------- @@ -1850,7 +2054,8 @@ namespace nodetool { std::vector<peerlist_entry> pl_white; std::vector<peerlist_entry> pl_gray; - m_peerlist.get_peerlist_full(pl_gray, pl_white); + for (auto& zone : m_network_zones) + zone.second.m_peerlist.get_peerlist(pl_gray, pl_white); MINFO(ENDL << "Peerlist white:" << ENDL << print_peerlist_to_string(pl_white) << ENDL << "Peerlist gray:" << ENDL << print_peerlist_to_string(pl_gray) ); return true; } @@ -1867,14 +2072,17 @@ namespace nodetool { std::stringstream ss; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + for (auto& zone : m_network_zones) { - ss << cntxt.m_remote_address.str() - << " \t\tpeer_id " << cntxt.peer_id - << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") - << std::endl; - return true; - }); + zone.second.m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + { + ss << cntxt.m_remote_address.str() + << " \t\tpeer_id " << cntxt.peer_id + << " \t\tconn_id " << cntxt.m_connection_id << (cntxt.m_is_income ? " INC":" OUT") + << std::endl; + return true; + }); + } std::string s = ss.str(); return s; } @@ -1888,11 +2096,12 @@ namespace nodetool template<class t_payload_net_handler> void node_server<t_payload_net_handler>::on_connection_close(p2p_connection_context& context) { - if (!m_net_server.is_stop_signal_sent() && !context.m_is_income) { + network_zone& zone = m_network_zones.at(context.m_remote_address.get_zone()); + if (!zone.m_net_server.is_stop_signal_sent() && !context.m_is_income) { epee::net_utils::network_address na = AUTO_VAL_INIT(na); na = context.m_remote_address; - m_peerlist.remove_from_peer_anchor(na); + zone.m_peerlist.remove_from_peer_anchor(na); } m_payload_handler.on_connection_close(context); @@ -1909,9 +2118,10 @@ namespace nodetool template<class t_payload_net_handler> template <class Container> bool node_server<t_payload_net_handler>::connect_to_peerlist(const Container& peers) { + const network_zone& public_zone = m_network_zones.at(epee::net_utils::zone::public_); for(const epee::net_utils::network_address& na: peers) { - if(m_net_server.is_stop_signal_sent()) + if(public_zone.m_net_server.is_stop_signal_sent()) return false; if(is_addr_connected(na)) @@ -1930,16 +2140,16 @@ namespace nodetool for(const std::string& pr_str: perrs) { - epee::net_utils::network_address na = AUTO_VAL_INIT(na); const uint16_t default_port = cryptonote::get_config(m_nettype).P2P_DEFAULT_PORT; - bool r = parse_peer_from_string(na, pr_str, default_port); - if (r) + expect<epee::net_utils::network_address> adr = net::get_network_address(pr_str, default_port); + if (adr) { - container.push_back(na); + add_zone(adr->get_zone()); + container.push_back(std::move(*adr)); continue; } std::vector<epee::net_utils::network_address> resolved_addrs; - r = append_net_address(resolved_addrs, pr_str, default_port); + bool r = append_net_address(resolved_addrs, pr_str, default_port); CHECK_AND_ASSERT_MES(r, false, "Failed to parse or resolve address from string: " << pr_str); for (const epee::net_utils::network_address& addr : resolved_addrs) { @@ -1951,32 +2161,47 @@ namespace nodetool } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_out_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_out_peers(network_zone& zone, int64_t max) { - if(max == -1) - max = P2P_DEFAULT_CONNECTIONS_COUNT; - m_config.m_net_config.max_out_connection_count = max; - m_payload_handler.set_max_out_peers(max); + if(max == -1) { + zone.m_config.m_net_config.max_out_connection_count = P2P_DEFAULT_CONNECTIONS_COUNT; + return true; + } + zone.m_config.m_net_config.max_out_connection_count = max; return true; } template<class t_payload_net_handler> - bool node_server<t_payload_net_handler>::set_max_in_peers(const boost::program_options::variables_map& vm, int64_t max) + bool node_server<t_payload_net_handler>::set_max_in_peers(network_zone& zone, int64_t max) { - m_config.m_net_config.max_in_connection_count = max; + zone.m_config.m_net_config.max_in_connection_count = max; return true; } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_out_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_out_public_peers(size_t count) { - m_net_server.get_config_object().del_out_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_out_connection_count; + public_zone->second.m_config.m_net_config.max_out_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_out_connections(current - count); + } } template<class t_payload_net_handler> - void node_server<t_payload_net_handler>::delete_in_connections(size_t count) + void node_server<t_payload_net_handler>::change_max_in_public_peers(size_t count) { - m_net_server.get_config_object().del_in_connections(count); + auto public_zone = m_network_zones.find(epee::net_utils::zone::public_); + if (public_zone != m_network_zones.end()) + { + const auto current = public_zone->second.m_config.m_net_config.max_in_connection_count; + public_zone->second.m_config.m_net_config.max_in_connection_count = count; + if(current > count) + public_zone->second.m_net_server.get_config_object().del_in_connections(current - count); + } } template<class t_payload_net_handler> @@ -2049,10 +2274,13 @@ namespace nodetool template<class t_payload_net_handler> bool node_server<t_payload_net_handler>::has_too_many_connections(const epee::net_utils::network_address &address) { + if (address.get_zone() != epee::net_utils::zone::public_) + return false; // Unable to determine how many connections from host + const size_t max_connections = 1; size_t count = 0; - m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) + m_network_zones.at(epee::net_utils::zone::public_).m_net_server.get_config_object().foreach_connection([&](const p2p_connection_context& cntxt) { if (cntxt.m_is_income && cntxt.m_remote_address.is_same_host(address)) { count++; @@ -2075,29 +2303,29 @@ namespace nodetool if (!m_exclusive_peers.empty()) return true; if (m_payload_handler.needs_new_sync_connections()) return true; - peerlist_entry pe = AUTO_VAL_INIT(pe); - - if (m_net_server.is_stop_signal_sent()) - return false; - - if (!m_peerlist.get_random_gray_peer(pe)) { - return true; - } - - bool success = check_connection_and_handshake_with_peer(pe.adr, pe.last_seen); + for (auto& zone : m_network_zones) + { + if (zone.second.m_net_server.is_stop_signal_sent()) + return false; - if (!success) { - m_peerlist.remove_from_peer_gray(pe); + if (zone.second.m_connect == nullptr) + continue; - LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + peerlist_entry pe{}; + if (!zone.second.m_peerlist.get_random_gray_peer(pe)) + continue; - return true; + if (!check_connection_and_handshake_with_peer(pe.adr, pe.last_seen)) + { + zone.second.m_peerlist.remove_from_peer_gray(pe); + LOG_PRINT_L2("PEER EVICTED FROM GRAY PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } + else + { + zone.second.m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); + LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); + } } - - m_peerlist.set_peer_just_seen(pe.id, pe.adr, pe.pruning_seed); - - LOG_PRINT_L2("PEER PROMOTED TO WHITE PEER LIST IP address: " << pe.adr.host_str() << " Peer ID: " << peerid_type(pe.id)); - return true; } @@ -2225,4 +2453,37 @@ namespace nodetool MINFO("No IGD was found."); } } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::socks_connect(network_zone& zone, const epee::net_utils::network_address& remote) + { + auto result = socks_connect_internal(zone.m_net_server.get_stop_signal(), zone.m_net_server.get_io_service(), zone.m_proxy_address, remote); + if (result) // if no error + { + p2p_connection_context context{}; + if (zone.m_net_server.add_connection(context, std::move(*result), remote)) + return {std::move(context)}; + } + return boost::none; + } + + template<typename t_payload_net_handler> + boost::optional<p2p_connection_context_t<typename t_payload_net_handler::connection_context>> + node_server<t_payload_net_handler>::public_connect(network_zone& zone, epee::net_utils::network_address const& na) + { + CHECK_AND_ASSERT_MES(na.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id(), boost::none, + "Only IPv4 addresses are supported here"); + const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); + + typename net_server::t_connection_context con{}; + const bool res = zone.m_net_server.connect(epee::string_tools::get_ip_string_from_int32(ipv4.ip()), + epee::string_tools::num_to_string_fast(ipv4.port()), + zone.m_config.m_net_config.connection_timeout, + con); + + if (res) + return {std::move(con)}; + return boost::none; + } } diff --git a/src/p2p/net_node_common.h b/src/p2p/net_node_common.h index 075c18ef9..944bf48e4 100644 --- a/src/p2p/net_node_common.h +++ b/src/p2p/net_node_common.h @@ -31,6 +31,8 @@ #pragma once #include <boost/uuid/uuid.hpp> +#include <utility> +#include <vector> #include "net/net_utils_base.h" #include "p2p_protocol_defs.h" @@ -43,13 +45,13 @@ namespace nodetool template<class t_connection_context> struct i_p2p_endpoint { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections)=0; - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context)=0; + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections)=0; virtual bool invoke_command_to_peer(int command, const epee::span<const uint8_t> req_buff, std::string& resp_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool invoke_notify_to_peer(int command, const epee::span<const uint8_t> req_buff, const epee::net_utils::connection_context_base& context)=0; virtual bool drop_connection(const epee::net_utils::connection_context_base& context)=0; virtual void request_callback(const epee::net_utils::connection_context_base& context)=0; - virtual uint64_t get_connections_count()=0; + virtual uint64_t get_public_connections_count()=0; + virtual size_t get_zone_count() const=0; virtual void for_each_connection(std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool for_connection(const boost::uuids::uuid&, std::function<bool(t_connection_context&, peerid_type, uint32_t)> f)=0; virtual bool block_host(const epee::net_utils::network_address &address, time_t seconds = 0)=0; @@ -64,11 +66,7 @@ namespace nodetool template<class t_connection_context> struct p2p_endpoint_stub: public i_p2p_endpoint<t_connection_context> { - virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, const std::list<boost::uuids::uuid>& connections) - { - return false; - } - virtual bool relay_notify_to_all(int command, const epee::span<const uint8_t> data_buff, const epee::net_utils::connection_context_base& context) + virtual bool relay_notify_to_list(int command, const epee::span<const uint8_t> data_buff, std::vector<std::pair<epee::net_utils::zone, boost::uuids::uuid>> connections) { return false; } @@ -97,7 +95,12 @@ namespace nodetool return false; } - virtual uint64_t get_connections_count() + virtual size_t get_zone_count() const + { + return 0; + } + + virtual uint64_t get_public_connections_count() { return false; } diff --git a/src/p2p/net_peerlist.cpp b/src/p2p/net_peerlist.cpp new file mode 100644 index 000000000..ce5c67fe5 --- /dev/null +++ b/src/p2p/net_peerlist.cpp @@ -0,0 +1,295 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include "net_peerlist.h" + +#include <algorithm> +#include <functional> +#include <fstream> +#include <iterator> + +#include <boost/archive/binary_iarchive.hpp> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/filesystem/operations.hpp> +#include <boost/range/join.hpp> +#include <boost/serialization/version.hpp> + +#include "net_peerlist_boost_serialization.h" + + +namespace nodetool +{ + namespace + { + constexpr unsigned CURRENT_PEERLIST_STORAGE_ARCHIVE_VER = 6; + + struct by_zone + { + using zone = epee::net_utils::zone; + + template<typename T> + bool operator()(const T& left, const zone right) const + { + return left.adr.get_zone() < right; + } + + template<typename T> + bool operator()(const zone left, const T& right) const + { + return left < right.adr.get_zone(); + } + + template<typename T, typename U> + bool operator()(const T& left, const U& right) const + { + return left.adr.get_zone() < right.adr.get_zone(); + } + }; + + template<typename Elem, typename Archive> + std::vector<Elem> load_peers(Archive& a, unsigned ver) + { + // at v6, we drop existing peerlists, because annoying change + if (ver < 6) + return {}; + + uint64_t size = 0; + a & size; + + Elem ple{}; + + std::vector<Elem> elems{}; + elems.reserve(size); + while (size--) + { + a & ple; + elems.push_back(std::move(ple)); + } + + return elems; + } + + template<typename Archive, typename Range> + void save_peers(Archive& a, const Range& elems) + { + const uint64_t size = elems.size(); + a & size; + for (const auto& elem : elems) + a & elem; + } + + template<typename T> + std::vector<T> do_take_zone(std::vector<T>& src, epee::net_utils::zone zone) + { + const auto start = std::lower_bound(src.begin(), src.end(), zone, by_zone{}); + const auto end = std::upper_bound(start, src.end(), zone, by_zone{}); + + std::vector<T> out{}; + out.assign(std::make_move_iterator(start), std::make_move_iterator(end)); + src.erase(start, end); + return out; + } + + template<typename Container, typename T> + void add_peers(Container& dest, std::vector<T>&& src) + { + dest.insert(std::make_move_iterator(src.begin()), std::make_move_iterator(src.end())); + } + + template<typename Container, typename Range> + void copy_peers(Container& dest, const Range& src) + { + std::copy(src.begin(), src.end(), std::back_inserter(dest)); + } + } // anonymous + + struct peerlist_join + { + const peerlist_types& ours; + const peerlist_types& other; + }; + + template<typename Archive> + void serialize(Archive& a, peerlist_types& elem, unsigned ver) + { + elem.white = load_peers<peerlist_entry>(a, ver); + elem.gray = load_peers<peerlist_entry>(a, ver); + elem.anchor = load_peers<anchor_peerlist_entry>(a, ver); + + if (ver == 0) + { + // from v1, we do not store the peer id anymore + peerid_type peer_id{}; + a & peer_id; + } + } + + template<typename Archive> + void serialize(Archive& a, peerlist_join elem, unsigned ver) + { + save_peers(a, boost::range::join(elem.ours.white, elem.other.white)); + save_peers(a, boost::range::join(elem.ours.gray, elem.other.gray)); + save_peers(a, boost::range::join(elem.ours.anchor, elem.other.anchor)); + } + + boost::optional<peerlist_storage> peerlist_storage::open(std::istream& src, const bool new_format) + { + try + { + peerlist_storage out{}; + if (new_format) + { + boost::archive::portable_binary_iarchive a{src}; + a >> out.m_types; + } + else + { + boost::archive::binary_iarchive a{src}; + a >> out.m_types; + } + + if (src.good()) + { + std::sort(out.m_types.white.begin(), out.m_types.white.end(), by_zone{}); + std::sort(out.m_types.gray.begin(), out.m_types.gray.end(), by_zone{}); + std::sort(out.m_types.anchor.begin(), out.m_types.anchor.end(), by_zone{}); + return {std::move(out)}; + } + } + catch (const std::exception& e) + {} + + return boost::none; + } + + boost::optional<peerlist_storage> peerlist_storage::open(const std::string& path) + { + std::ifstream src_file{}; + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + boost::optional<peerlist_storage> out = open(src_file, true); + if (!out) + { + // if failed, try reading in unportable mode + boost::filesystem::copy_file(path, path + ".unportable", boost::filesystem::copy_option::overwrite_if_exists); + src_file.close(); + src_file.open( path , std::ios_base::binary | std::ios_base::in); + if(src_file.fail()) + return boost::none; + + out = open(src_file, false); + if (!out) + { + // This is different from the `return boost::none` cases above. Those + // cases could fail due to bad file permissions, so a shutdown is + // likely more appropriate. + MWARNING("Failed to load p2p config file, falling back to default config"); + out.emplace(); + } + } + + return out; + } + + peerlist_storage::~peerlist_storage() noexcept + {} + + bool peerlist_storage::store(std::ostream& dest, const peerlist_types& other) const + { + try + { + boost::archive::portable_binary_oarchive a{dest}; + const peerlist_join pj{std::cref(m_types), std::cref(other)}; + a << pj; + return dest.good(); + } + catch (const boost::archive::archive_exception& e) + {} + + return false; + } + + bool peerlist_storage::store(const std::string& path, const peerlist_types& other) const + { + std::ofstream dest_file{}; + dest_file.open( path , std::ios_base::binary | std::ios_base::out| std::ios::trunc); + if(dest_file.fail()) + return false; + + return store(dest_file, other); + } + + peerlist_types peerlist_storage::take_zone(epee::net_utils::zone zone) + { + peerlist_types out{}; + out.white = do_take_zone(m_types.white, zone); + out.gray = do_take_zone(m_types.gray, zone); + out.anchor = do_take_zone(m_types.anchor, zone); + return out; + } + + bool peerlist_manager::init(peerlist_types&& peers, bool allow_local_ip) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + + if (!m_peers_white.empty() || !m_peers_gray.empty() || !m_peers_anchor.empty()) + return false; + + add_peers(m_peers_white.get<by_addr>(), std::move(peers.white)); + add_peers(m_peers_gray.get<by_addr>(), std::move(peers.gray)); + add_peers(m_peers_anchor.get<by_addr>(), std::move(peers.anchor)); + m_allow_local_ip = allow_local_ip; + return true; + } + + void peerlist_manager::get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + copy_peers(pl_gray, m_peers_gray.get<by_addr>()); + copy_peers(pl_white, m_peers_white.get<by_addr>()); + } + + void peerlist_manager::get_peerlist(peerlist_types& peers) + { + CRITICAL_REGION_LOCAL(m_peerlist_lock); + peers.white.reserve(peers.white.size() + m_peers_white.size()); + peers.gray.reserve(peers.gray.size() + m_peers_gray.size()); + peers.anchor.reserve(peers.anchor.size() + m_peers_anchor.size()); + + copy_peers(peers.white, m_peers_white.get<by_addr>()); + copy_peers(peers.gray, m_peers_gray.get<by_addr>()); + copy_peers(peers.anchor, m_peers_anchor.get<by_addr>()); + } +} + +BOOST_CLASS_VERSION(nodetool::peerlist_types, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); +BOOST_CLASS_VERSION(nodetool::peerlist_join, nodetool::CURRENT_PEERLIST_STORAGE_ARCHIVE_VER); + diff --git a/src/p2p/net_peerlist.h b/src/p2p/net_peerlist.h index 685fdc193..46726d7d8 100644 --- a/src/p2p/net_peerlist.h +++ b/src/p2p/net_peerlist.h @@ -30,33 +30,67 @@ #pragma once +#include <iosfwd> #include <list> -#include <set> -#include <map> -#include <boost/archive/binary_iarchive.hpp> -#include <boost/archive/portable_binary_oarchive.hpp> -#include <boost/archive/portable_binary_iarchive.hpp> -#include <boost/serialization/version.hpp> +#include <string> +#include <vector> #include <boost/multi_index_container.hpp> #include <boost/multi_index/ordered_index.hpp> #include <boost/multi_index/identity.hpp> #include <boost/multi_index/member.hpp> +#include <boost/optional/optional.hpp> #include <boost/range/adaptor/reversed.hpp> -#include "syncobj.h" +#include "cryptonote_config.h" +#include "net/enums.h" #include "net/local_ip.h" #include "p2p_protocol_defs.h" -#include "cryptonote_config.h" -#include "net_peerlist_boost_serialization.h" - - -#define CURRENT_PEERLIST_STORAGE_ARCHIVE_VER 6 +#include "syncobj.h" namespace nodetool { + struct peerlist_types + { + std::vector<peerlist_entry> white; + std::vector<peerlist_entry> gray; + std::vector<anchor_peerlist_entry> anchor; + }; + + class peerlist_storage + { + public: + peerlist_storage() + : m_types{} + {} + + //! \return Peers stored in stream `src` in `new_format` (portable archive or older non-portable). + static boost::optional<peerlist_storage> open(std::istream& src, const bool new_format); + + //! \return Peers stored in file at `path` + static boost::optional<peerlist_storage> open(const std::string& path); + + peerlist_storage(peerlist_storage&&) = default; + peerlist_storage(const peerlist_storage&) = delete; + + ~peerlist_storage() noexcept; + peerlist_storage& operator=(peerlist_storage&&) = default; + peerlist_storage& operator=(const peerlist_storage&) = delete; + + //! Save peers from `this` and `other` in stream `dest`. + bool store(std::ostream& dest, const peerlist_types& other) const; + + //! Save peers from `this` and `other` in one file at `path`. + bool store(const std::string& path, const peerlist_types& other) const; + + //! \return Peers in `zone` and from remove from `this`. + peerlist_types take_zone(epee::net_utils::zone zone); + + private: + peerlist_types m_types; + }; /************************************************************************/ /* */ @@ -64,13 +98,13 @@ namespace nodetool class peerlist_manager { public: - bool init(bool allow_local_ip); - bool deinit(); + bool init(peerlist_types&& peers, bool allow_local_ip); size_t get_white_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_white.size();} size_t get_gray_peers_count(){CRITICAL_REGION_LOCAL(m_peerlist_lock); return m_peers_gray.size();} bool merge_peerlist(const std::vector<peerlist_entry>& outer_bs); bool get_peerlist_head(std::vector<peerlist_entry>& bs_head, uint32_t depth = P2P_DEFAULT_PEERS_IN_HANDSHAKE); - bool get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white); + void get_peerlist(peerlist_types& peers); bool get_white_peer_by_index(peerlist_entry& p, size_t i); bool get_gray_peer_by_index(peerlist_entry& p, size_t i); template<typename F> bool foreach(bool white, const F &f); @@ -136,18 +170,6 @@ namespace nodetool > peers_indexed; typedef boost::multi_index_container< - peerlist_entry, - boost::multi_index::indexed_by< - // access by peerlist_entry::id< - boost::multi_index::ordered_unique<boost::multi_index::tag<by_id>, boost::multi_index::member<peerlist_entry,uint64_t,&peerlist_entry::id> >, - // access by peerlist_entry::net_adress - boost::multi_index::ordered_unique<boost::multi_index::tag<by_addr>, boost::multi_index::member<peerlist_entry,epee::net_utils::network_address,&peerlist_entry::adr> >, - // sort by peerlist_entry::last_seen< - boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<peerlist_entry,int64_t,&peerlist_entry::last_seen> > - > - > peers_indexed_old; - - typedef boost::multi_index_container< anchor_peerlist_entry, boost::multi_index::indexed_by< // access by anchor_peerlist_entry::net_adress @@ -156,56 +178,8 @@ namespace nodetool boost::multi_index::ordered_non_unique<boost::multi_index::tag<by_time>, boost::multi_index::member<anchor_peerlist_entry,int64_t,&anchor_peerlist_entry::first_seen> > > > anchor_peers_indexed; - public: - - template <class Archive, class List, class Element, class t_version_type> - void serialize_peers(Archive &a, List &list, Element ple, const t_version_type ver) - { - if (typename Archive::is_saving()) - { - uint64_t size = list.size(); - a & size; - for (auto p: list) - { - a & p; - } - } - else - { - uint64_t size; - a & size; - list.clear(); - while (size--) - { - a & ple; - list.insert(ple); - } - } - } - - template <class Archive, class t_version_type> - void serialize(Archive &a, const t_version_type ver) - { - // at v6, we drop existing peerlists, because annoying change - if (ver < 6) - return; - - CRITICAL_REGION_LOCAL(m_peerlist_lock); - -#if 0 - // trouble loading more than one peer, can't find why - a & m_peers_white; - a & m_peers_gray; - a & m_peers_anchor; -#else - serialize_peers(a, m_peers_white, peerlist_entry(), ver); - serialize_peers(a, m_peers_gray, peerlist_entry(), ver); - serialize_peers(a, m_peers_anchor, anchor_peerlist_entry(), ver); -#endif - } private: - bool peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi); void trim_white_peerlist(); void trim_gray_peerlist(); @@ -220,34 +194,6 @@ namespace nodetool anchor_peers_indexed m_peers_anchor; }; //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::init(bool allow_local_ip) - { - m_allow_local_ip = allow_local_ip; - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::deinit() - { - return true; - } - //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::peers_indexed_from_old(const peers_indexed_old& pio, peers_indexed& pi) - { - for(auto x: pio) - { - auto by_addr_it = pi.get<by_addr>().find(x.adr); - if(by_addr_it == pi.get<by_addr>().end()) - { - pi.insert(x); - } - } - - return true; - } - //-------------------------------------------------------------------------------------------------- inline void peerlist_manager::trim_gray_peerlist() { while(m_peers_gray.size() > P2P_LOCAL_GRAY_PEERLIST_LIMIT) @@ -337,27 +283,6 @@ namespace nodetool return true; } //-------------------------------------------------------------------------------------------------- - inline - bool peerlist_manager::get_peerlist_full(std::vector<peerlist_entry>& pl_gray, std::vector<peerlist_entry>& pl_white) - { - CRITICAL_REGION_LOCAL(m_peerlist_lock); - peers_indexed::index<by_time>::type& by_time_index_gr=m_peers_gray.get<by_time>(); - pl_gray.resize(pl_gray.size() + by_time_index_gr.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_gr)) - { - pl_gray.push_back(vl); - } - - peers_indexed::index<by_time>::type& by_time_index_wt=m_peers_white.get<by_time>(); - pl_white.resize(pl_white.size() + by_time_index_wt.size()); - for(const peers_indexed::value_type& vl: boost::adaptors::reverse(by_time_index_wt)) - { - pl_white.push_back(vl); - } - - return true; - } - //-------------------------------------------------------------------------------------------------- template<typename F> inline bool peerlist_manager::foreach(bool white, const F &f) { @@ -559,4 +484,3 @@ namespace nodetool //-------------------------------------------------------------------------------------------------- } -BOOST_CLASS_VERSION(nodetool::peerlist_manager, CURRENT_PEERLIST_STORAGE_ARCHIVE_VER) diff --git a/src/p2p/net_peerlist_boost_serialization.h b/src/p2p/net_peerlist_boost_serialization.h index 2f4a7e661..d2e9efa3d 100644 --- a/src/p2p/net_peerlist_boost_serialization.h +++ b/src/p2p/net_peerlist_boost_serialization.h @@ -1,4 +1,4 @@ -// Copyright (c) 2014-2018, The Monero Project + // Copyright (c) 2014-2018, The Monero Project // // All rights reserved. // @@ -30,7 +30,11 @@ #pragma once +#include <cstring> + +#include "common/expect.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" #include "p2p/p2p_protocol_defs.h" #ifdef CRYPTONOTE_PRUNING_DEBUG_SPOOF_SEED @@ -42,24 +46,37 @@ namespace boost namespace serialization { template <class T, class Archive> - inline void do_serialize(Archive &a, epee::net_utils::network_address& na, T local) + inline void do_serialize(boost::mpl::false_, Archive &a, epee::net_utils::network_address& na) { - if (typename Archive::is_saving()) local = na.as<T>(); - a & local; - if (!typename Archive::is_saving()) na = local; + T addr{}; + a & addr; + na = std::move(addr); } + + template <class T, class Archive> + inline void do_serialize(boost::mpl::true_, Archive &a, const epee::net_utils::network_address& na) + { + a & na.as<T>(); + } + template <class Archive, class ver_type> inline void serialize(Archive &a, epee::net_utils::network_address& na, const ver_type ver) { + static constexpr const typename Archive::is_saving is_saving{}; + uint8_t type; - if (typename Archive::is_saving()) - type = na.get_type_id(); + if (is_saving) + type = uint8_t(na.get_type_id()); a & type; - switch (type) + switch (epee::net_utils::address_type(type)) { - case epee::net_utils::ipv4_network_address::ID: - do_serialize(a, na, epee::net_utils::ipv4_network_address{0, 0}); - break; + case epee::net_utils::ipv4_network_address::get_type_id(): + do_serialize<epee::net_utils::ipv4_network_address>(is_saving, a, na); + break; + case net::tor_address::get_type_id(): + do_serialize<net::tor_address>(is_saving, a, na); + break; + case epee::net_utils::address_type::invalid: default: throw std::runtime_error("Unsupported network address type"); } @@ -76,6 +93,47 @@ namespace boost } template <class Archive, class ver_type> + inline void save(Archive& a, const net::tor_address& na, const ver_type) + { + const size_t length = std::strlen(na.host_str()); + if (length > 255) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + const uint16_t port{na.port()}; + const uint8_t len = length; + a & port; + a & len; + a.save_binary(na.host_str(), length); + } + + template <class Archive, class ver_type> + inline void load(Archive& a, net::tor_address& na, const ver_type) + { + uint16_t port = 0; + uint8_t length = 0; + a & port; + a & length; + + if (length > net::tor_address::buffer_size()) + MONERO_THROW(net::error::invalid_tor_address, "Tor address too long"); + + char host[net::tor_address::buffer_size()] = {0}; + a.load_binary(host, length); + host[sizeof(host) - 1] = 0; + + if (std::strcmp(host, net::tor_address::unknown_str()) == 0) + na = net::tor_address::unknown(); + else + na = MONERO_UNWRAP(net::tor_address::make(host, port)); + } + + template <class Archive, class ver_type> + inline void serialize(Archive &a, net::tor_address& na, const ver_type ver) + { + boost::serialization::split_free(a, na, ver); + } + + template <class Archive, class ver_type> inline void serialize(Archive &a, nodetool::peerlist_entry& pl, const ver_type ver) { a & pl.adr; diff --git a/src/p2p/p2p_protocol_defs.h b/src/p2p/p2p_protocol_defs.h index 939cedf42..6e5e45008 100644 --- a/src/p2p/p2p_protocol_defs.h +++ b/src/p2p/p2p_protocol_defs.h @@ -34,6 +34,7 @@ #include <boost/serialization/version.hpp> #include "serialization/keyvalue_serialization.h" #include "net/net_utils_base.h" +#include "net/tor_address.h" // needed for serialization #include "misc_language.h" #include "string_tools.h" #include "time_helper.h" @@ -204,7 +205,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); @@ -263,7 +264,7 @@ namespace nodetool std::vector<peerlist_entry_base<network_address_old>> local_peerlist; for (const auto &p: this_ref.local_peerlist_new) { - if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (p.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) { const epee::net_utils::network_address &na = p.adr; const epee::net_utils::ipv4_network_address &ipv4 = na.as<const epee::net_utils::ipv4_network_address>(); diff --git a/src/rpc/CMakeLists.txt b/src/rpc/CMakeLists.txt index d2c4a33cb..60cae036c 100644 --- a/src/rpc/CMakeLists.txt +++ b/src/rpc/CMakeLists.txt @@ -112,6 +112,7 @@ target_link_libraries(rpc common cryptonote_core cryptonote_protocol + net version ${Boost_REGEX_LIBRARY} ${Boost_THREAD_LIBRARY} diff --git a/src/rpc/core_rpc_server.cpp b/src/rpc/core_rpc_server.cpp index d308dd63d..b524273bf 100644 --- a/src/rpc/core_rpc_server.cpp +++ b/src/rpc/core_rpc_server.cpp @@ -42,6 +42,7 @@ using namespace epee; #include "cryptonote_basic/account.h" #include "cryptonote_basic/cryptonote_basic_impl.h" #include "misc_language.h" +#include "net/parse.h" #include "storages/http_abstract_invoke.h" #include "crypto/hash.h" #include "rpc/rpc_args.h" @@ -185,12 +186,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -902,12 +903,12 @@ namespace cryptonote PERF_TIMER(on_get_peer_list); std::vector<nodetool::peerlist_entry> white_list; std::vector<nodetool::peerlist_entry> gray_list; - m_p2p.get_peerlist_manager().get_peerlist_full(gray_list, white_list); + m_p2p.get_public_peerlist(gray_list, white_list); res.white_list.reserve(white_list.size()); for (auto & entry : white_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.white_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -917,7 +918,7 @@ namespace cryptonote res.gray_list.reserve(gray_list.size()); for (auto & entry : gray_list) { - if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::ID) + if (entry.adr.get_type_id() == epee::net_utils::ipv4_network_address::get_type_id()) res.gray_list.emplace_back(entry.id, entry.adr.as<epee::net_utils::ipv4_network_address>().ip(), entry.adr.as<epee::net_utils::ipv4_network_address>().port(), entry.last_seen, entry.pruning_seed); else @@ -1646,12 +1647,12 @@ namespace cryptonote res.tx_count = m_core.get_blockchain_storage().get_total_transactions() - res.height; //without coinbase res.tx_pool_size = m_core.get_pool_transactions_count(); res.alt_blocks_count = restricted ? 0 : m_core.get_blockchain_storage().get_alternative_blocks_count(); - uint64_t total_conn = restricted ? 0 : m_p2p.get_connections_count(); - res.outgoing_connections_count = restricted ? 0 : m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = restricted ? 0 : m_p2p.get_public_connections_count(); + res.outgoing_connections_count = restricted ? 0 : m_p2p.get_public_outgoing_connections_count(); res.incoming_connections_count = restricted ? 0 : (total_conn - res.outgoing_connections_count); res.rpc_connections_count = restricted ? 0 : get_connections_count(); - res.white_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_white_peers_count(); - res.grey_peerlist_size = restricted ? 0 : m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.white_peerlist_size = restricted ? 0 : m_p2p.get_public_white_peers_count(); + res.grey_peerlist_size = restricted ? 0 : m_p2p.get_public_gray_peers_count(); cryptonote::network_type net_type = nettype(); res.mainnet = net_type == MAINNET; @@ -1730,12 +1731,14 @@ namespace cryptonote epee::net_utils::network_address na; if (!i->host.empty()) { - if (!epee::net_utils::create_network_address(na, i->host)) + auto na_parsed = net::get_network_address(i->host, 0); + if (!na_parsed) { error_resp.code = CORE_RPC_ERROR_CODE_WRONG_PARAM; error_resp.message = "Unsupported host type"; return false; } + na = std::move(*na_parsed); } else { @@ -1958,11 +1961,7 @@ namespace cryptonote bool core_rpc_server::on_out_peers(const COMMAND_RPC_OUT_PEERS::request& req, COMMAND_RPC_OUT_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_out_peers); - size_t n_connections = m_p2p.get_outgoing_connections_count(); - size_t n_delete = (n_connections > req.out_peers) ? n_connections - req.out_peers : 0; - m_p2p.m_config.m_net_config.max_out_connection_count = req.out_peers; - if (n_delete) - m_p2p.delete_out_connections(n_delete); + m_p2p.change_max_out_public_peers(req.out_peers); res.status = CORE_RPC_STATUS_OK; return true; } @@ -1970,11 +1969,7 @@ namespace cryptonote bool core_rpc_server::on_in_peers(const COMMAND_RPC_IN_PEERS::request& req, COMMAND_RPC_IN_PEERS::response& res, const connection_context *ctx) { PERF_TIMER(on_in_peers); - size_t n_connections = m_p2p.get_incoming_connections_count(); - size_t n_delete = (n_connections > req.in_peers) ? n_connections - req.in_peers : 0; - m_p2p.m_config.m_net_config.max_in_connection_count = req.in_peers; - if (n_delete) - m_p2p.delete_in_connections(n_delete); + m_p2p.change_max_in_public_peers(req.in_peers); res.status = CORE_RPC_STATUS_OK; return true; } diff --git a/src/rpc/daemon_handler.cpp b/src/rpc/daemon_handler.cpp index e2885dbb5..871f7d368 100644 --- a/src/rpc/daemon_handler.cpp +++ b/src/rpc/daemon_handler.cpp @@ -423,13 +423,13 @@ namespace rpc res.info.alt_blocks_count = chain.get_alternative_blocks_count(); - uint64_t total_conn = m_p2p.get_connections_count(); - res.info.outgoing_connections_count = m_p2p.get_outgoing_connections_count(); + uint64_t total_conn = m_p2p.get_public_connections_count(); + res.info.outgoing_connections_count = m_p2p.get_public_outgoing_connections_count(); res.info.incoming_connections_count = total_conn - res.info.outgoing_connections_count; - res.info.white_peerlist_size = m_p2p.get_peerlist_manager().get_white_peers_count(); + res.info.white_peerlist_size = m_p2p.get_public_white_peers_count(); - res.info.grey_peerlist_size = m_p2p.get_peerlist_manager().get_gray_peers_count(); + res.info.grey_peerlist_size = m_p2p.get_public_gray_peers_count(); res.info.mainnet = m_core.get_nettype() == MAINNET; res.info.testnet = m_core.get_nettype() == TESTNET; diff --git a/tests/unit_tests/CMakeLists.txt b/tests/unit_tests/CMakeLists.txt index 5f7fa84a5..cb421c847 100644 --- a/tests/unit_tests/CMakeLists.txt +++ b/tests/unit_tests/CMakeLists.txt @@ -62,6 +62,7 @@ set(unit_tests_sources mul_div.cpp multiexp.cpp multisig.cpp + net.cpp notify.cpp output_distribution.cpp parse_amount.cpp @@ -100,6 +101,7 @@ target_link_libraries(unit_tests cryptonote_core blockchain_db rpc + net serialization wallet p2p diff --git a/tests/unit_tests/epee_utils.cpp b/tests/unit_tests/epee_utils.cpp index 3d5882d7d..18fb262c2 100644 --- a/tests/unit_tests/epee_utils.cpp +++ b/tests/unit_tests/epee_utils.cpp @@ -545,6 +545,8 @@ TEST(StringTools, GetIpInt32) TEST(NetUtils, IPv4NetworkAddress) { + static_assert(epee::net_utils::ipv4_network_address::get_type_id() == epee::net_utils::address_type::ipv4, "bad ipv4 type id"); + const auto ip1 = boost::endian::native_to_big(0x330012FFu); const auto ip_loopback = boost::endian::native_to_big(0x7F000001u); const auto ip_local = boost::endian::native_to_big(0x0A000000u); @@ -555,7 +557,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip1, address1.ip()); EXPECT_EQ(65535, address1.port()); EXPECT_TRUE(epee::net_utils::ipv4_network_address{std::move(address1)} == address1); @@ -568,7 +570,7 @@ TEST(NetUtils, IPv4NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); EXPECT_EQ(ip_loopback, loopback.ip()); EXPECT_EQ(0, loopback.port()); @@ -624,7 +626,9 @@ TEST(NetUtils, NetworkAddress) constexpr static bool is_local() noexcept { return false; } static std::string str() { return {}; } static std::string host_str() { return {}; } - constexpr static uint8_t get_type_id() noexcept { return uint8_t(-1); } + constexpr static epee::net_utils::address_type get_type_id() noexcept { return epee::net_utils::address_type(-1); } + constexpr static epee::net_utils::zone get_zone() noexcept { return epee::net_utils::zone::invalid; } + constexpr static bool is_blockable() noexcept { return false; } }; const epee::net_utils::network_address empty; @@ -634,7 +638,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("<none>", empty.host_str().c_str()); EXPECT_FALSE(empty.is_loopback()); EXPECT_FALSE(empty.is_local()); - EXPECT_EQ(0, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::invalid, empty.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::invalid, empty.get_zone()); + EXPECT_FALSE(empty.is_blockable()); EXPECT_THROW(empty.as<custom_address>(), std::bad_cast); epee::net_utils::network_address address1{ @@ -650,7 +656,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("51.0.18.255", address1.host_str().c_str()); EXPECT_FALSE(address1.is_loopback()); EXPECT_FALSE(address1.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_TRUE(address1.is_blockable()); EXPECT_NO_THROW(address1.as<epee::net_utils::ipv4_network_address>()); EXPECT_THROW(address1.as<custom_address>(), std::bad_cast); @@ -667,7 +675,9 @@ TEST(NetUtils, NetworkAddress) EXPECT_STREQ("127.0.0.1", loopback.host_str().c_str()); EXPECT_TRUE(loopback.is_loopback()); EXPECT_FALSE(loopback.is_local()); - EXPECT_EQ(epee::net_utils::ipv4_network_address::ID, address1.get_type_id()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::public_, address1.get_zone()); + EXPECT_EQ(epee::net_utils::ipv4_network_address::get_type_id(), address1.get_type_id()); const epee::net_utils::network_address local{ epee::net_utils::ipv4_network_address{ip_local, 8080} diff --git a/tests/unit_tests/net.cpp b/tests/unit_tests/net.cpp new file mode 100644 index 000000000..a38ecfe81 --- /dev/null +++ b/tests/unit_tests/net.cpp @@ -0,0 +1,745 @@ +// Copyright (c) 2018, The Monero Project +// +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without modification, are +// permitted provided that the following conditions are met: +// +// 1. Redistributions of source code must retain the above copyright notice, this list of +// conditions and the following disclaimer. +// +// 2. Redistributions in binary form must reproduce the above copyright notice, this list +// of conditions and the following disclaimer in the documentation and/or other +// materials provided with the distribution. +// +// 3. Neither the name of the copyright holder nor the names of its contributors may be +// used to endorse or promote products derived from this software without specific +// prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY +// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL +// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF +// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +#include <atomic> +#include <boost/archive/portable_binary_oarchive.hpp> +#include <boost/archive/portable_binary_iarchive.hpp> +#include <boost/asio/buffer.hpp> +#include <boost/asio/io_service.hpp> +#include <boost/asio/ip/tcp.hpp> +#include <boost/asio/read.hpp> +#include <boost/asio/write.hpp> +#include <boost/endian/conversion.hpp> +#include <boost/system/error_code.hpp> +#include <boost/thread/thread.hpp> +#include <cstring> +#include <functional> +#include <gtest/gtest.h> +#include <memory> + +#include "net/error.h" +#include "net/net_utils_base.h" +#include "net/socks.h" +#include "net/parse.h" +#include "net/tor_address.h" +#include "p2p/net_peerlist_boost_serialization.h" +#include "serialization/keyvalue_serialization.h" +#include "storages/portable_storage.h" + +namespace +{ + static constexpr const char v2_onion[] = + "xmrto2bturnore26.onion"; + static constexpr const char v3_onion[] = + "vww6ybal4bd7szmgncyruucpgfkqahzddi37ktceo3ah7ngmcopnpyyd.onion"; +} + +TEST(tor_address, constants) +{ + static_assert(!net::tor_address::is_local(), "bad is_local() response"); + static_assert(!net::tor_address::is_loopback(), "bad is_loopback() response"); + static_assert(net::tor_address::get_type_id() == epee::net_utils::address_type::tor, "bad get_type_id() response"); + + EXPECT_FALSE(net::tor_address::is_local()); + EXPECT_FALSE(net::tor_address::is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, net::tor_address::get_type_id()); +} + +TEST(tor_address, invalid) +{ + EXPECT_TRUE(net::tor_address::make("").has_error()); + EXPECT_TRUE(net::tor_address::make(":").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion").has_error()); + EXPECT_TRUE(net::tor_address::make(".onion:").has_error()); + EXPECT_TRUE(net::tor_address::make(v2_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(v3_onion + 1).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v2_onion, sizeof(v2_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(boost::string_ref{v3_onion, sizeof(v3_onion) - 2}).has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":-").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v2_onion} + ":900a").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":65536").has_error()); + EXPECT_TRUE(net::tor_address::make(std::string{v3_onion} + ":-1").has_error()); + + std::string onion{v3_onion}; + onion.at(10) = 1; + EXPECT_TRUE(net::tor_address::make(onion).has_error()); +} + +TEST(tor_address, unblockable_types) +{ + net::tor_address tor{}; + + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + tor = net::tor_address::unknown(); + ASSERT_NE(nullptr, tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.host_str()); + EXPECT_STREQ("<unknown tor host>", tor.str().c_str()); + EXPECT_EQ(0u, tor.port()); + EXPECT_TRUE(tor.is_unknown()); + EXPECT_FALSE(tor.is_local()); + EXPECT_FALSE(tor.is_loopback()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor.get_zone()); + + EXPECT_EQ(net::tor_address{}, net::tor_address::unknown()); +} + +TEST(tor_address, valid) +{ + const auto address1 = net::tor_address::make(v3_onion); + + ASSERT_TRUE(address1.has_value()); + EXPECT_EQ(0u, address1->port()); + EXPECT_STREQ(v3_onion, address1->host_str()); + EXPECT_STREQ(v3_onion, address1->str().c_str()); + EXPECT_TRUE(address1->is_blockable()); + + net::tor_address address2{*address1}; + + EXPECT_EQ(0u, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_STREQ(v3_onion, address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_TRUE(address2.equal(*address1)); + EXPECT_TRUE(address1->equal(address2)); + EXPECT_TRUE(address2 == *address1); + EXPECT_TRUE(*address1 == address2); + EXPECT_FALSE(address2 != *address1); + EXPECT_FALSE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_FALSE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v2_onion} + ":6545")); + + EXPECT_EQ(6545, address2.port()); + EXPECT_STREQ(v2_onion, address2.host_str()); + EXPECT_EQ(std::string{v2_onion} + ":6545", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_FALSE(address2.is_same_host(*address1)); + EXPECT_FALSE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); + + address2 = MONERO_UNWRAP(net::tor_address::make(std::string{v3_onion} + ":", 65535)); + + EXPECT_EQ(65535, address2.port()); + EXPECT_STREQ(v3_onion, address2.host_str()); + EXPECT_EQ(std::string{v3_onion} + ":65535", address2.str().c_str()); + EXPECT_TRUE(address2.is_blockable()); + EXPECT_FALSE(address2.equal(*address1)); + EXPECT_FALSE(address1->equal(address2)); + EXPECT_FALSE(address2 == *address1); + EXPECT_FALSE(*address1 == address2); + EXPECT_TRUE(address2 != *address1); + EXPECT_TRUE(*address1 != address2); + EXPECT_TRUE(address2.is_same_host(*address1)); + EXPECT_TRUE(address1->is_same_host(address2)); + EXPECT_FALSE(address2.less(*address1)); + EXPECT_TRUE(address1->less(address2)); +} + +TEST(tor_address, generic_network_address) +{ + const epee::net_utils::network_address tor1{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address tor2{MONERO_UNWRAP(net::tor_address::make(v3_onion, 8080))}; + const epee::net_utils::network_address ip{epee::net_utils::ipv4_network_address{100, 200}}; + + EXPECT_EQ(tor1, tor2); + EXPECT_NE(ip, tor1); + EXPECT_LT(ip, tor1); + + EXPECT_STREQ(v3_onion, tor1.host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":8080", tor1.str()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor1.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::tor, tor2.get_type_id()); + EXPECT_EQ(epee::net_utils::address_type::ipv4, ip.get_type_id()); + EXPECT_EQ(epee::net_utils::zone::tor, tor1.get_zone()); + EXPECT_EQ(epee::net_utils::zone::tor, tor2.get_zone()); + EXPECT_EQ(epee::net_utils::zone::public_, ip.get_zone()); + EXPECT_TRUE(tor1.is_blockable()); + EXPECT_TRUE(tor2.is_blockable()); + EXPECT_TRUE(ip.is_blockable()); +} + +namespace +{ + struct test_command + { + net::tor_address tor; + + BEGIN_KV_SERIALIZE_MAP() + KV_SERIALIZE(tor); + END_KV_SERIALIZE_MAP() + }; +} + +TEST(tor_address, epee_serializev_v2) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v2_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v2_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v2_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serializev_v3) +{ + std::string buffer{}; + { + test_command command{MONERO_UNWRAP(net::tor_address::make(v3_onion, 10))}; + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_FALSE(command.tor.is_unknown()); + EXPECT_NE(net::tor_address{}, command.tor); + EXPECT_STREQ(v3_onion, command.tor.host_str()); + EXPECT_EQ(10u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(v3_onion), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, epee_serialize_unknown) +{ + std::string buffer{}; + { + test_command command{net::tor_address::unknown()}; + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(command.store(stg)); + EXPECT_TRUE(stg.store_to_binary(buffer)); + } + + test_command command{}; + { + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + epee::serialization::portable_storage stg{}; + EXPECT_TRUE(stg.load_from_binary(buffer)); + EXPECT_TRUE(command.load(stg)); + } + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STREQ(net::tor_address::unknown_str(), command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); + + // make sure that exceeding max buffer doesn't destroy tor_address::_load + { + epee::serialization::portable_storage stg{}; + stg.load_from_binary(buffer); + + std::string host{}; + ASSERT_TRUE(stg.get_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_EQ(std::strlen(net::tor_address::unknown_str()), host.size()); + + host.push_back('k'); + EXPECT_TRUE(stg.set_value("host", host, stg.open_section("tor", nullptr, false))); + EXPECT_TRUE(command.load(stg)); // poor error reporting from `KV_SERIALIZE` + } + + EXPECT_TRUE(command.tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, command.tor); + EXPECT_STRNE(v3_onion, command.tor.host_str()); + EXPECT_EQ(0u, command.tor.port()); +} + +TEST(tor_address, boost_serialize_v2) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v2_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v2_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_v3) +{ + std::string buffer{}; + { + const net::tor_address tor = MONERO_UNWRAP(net::tor_address::make(v3_onion, 10)); + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_FALSE(tor.is_unknown()); + EXPECT_NE(net::tor_address{}, tor); + EXPECT_STREQ(v3_onion, tor.host_str()); + EXPECT_EQ(10u, tor.port()); +} + +TEST(tor_address, boost_serialize_unknown) +{ + std::string buffer{}; + { + const net::tor_address tor{}; + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::ostringstream stream{}; + { + boost::archive::portable_binary_oarchive archive{stream}; + archive << tor; + } + buffer = stream.str(); + } + + net::tor_address tor{}; + { + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address{}, tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); + + std::istringstream stream{buffer}; + boost::archive::portable_binary_iarchive archive{stream}; + archive >> tor; + } + EXPECT_TRUE(tor.is_unknown()); + EXPECT_EQ(net::tor_address::unknown(), tor); + EXPECT_STREQ(net::tor_address::unknown_str(), tor.host_str()); + EXPECT_EQ(0u, tor.port()); +} + +TEST(get_network_address, onion) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("onion", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address(".onion", 0); + EXPECT_EQ(net::error::invalid_tor_address, address); + + address = net::get_network_address(v3_onion, 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":1000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::tor, address->get_type_id()); + EXPECT_STREQ(v3_onion, address->host_str().c_str()); + EXPECT_EQ(std::string{v3_onion} + ":2000", address->str()); + + address = net::get_network_address(std::string{v3_onion} + ":65536", 1000); + EXPECT_EQ(net::error::invalid_port, address); +} + + +TEST(get_network_address, ipv4) +{ + expect<epee::net_utils::network_address> address = + net::get_network_address("0.0.0.", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.257", 0); + EXPECT_EQ(net::error::unsupported_address, address); + + address = net::get_network_address("0.0.0.254", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("0.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("0.0.0.254:1000", address->str().c_str()); + + address = net::get_network_address("23.0.0.254:2000", 1000); + ASSERT_TRUE(bool(address)); + EXPECT_EQ(epee::net_utils::address_type::ipv4, address->get_type_id()); + EXPECT_STREQ("23.0.0.254", address->host_str().c_str()); + EXPECT_STREQ("23.0.0.254:2000", address->str().c_str()); +} + +namespace +{ + using stream_type = boost::asio::ip::tcp; + + struct io_thread + { + boost::asio::io_service io_service; + boost::asio::io_service::work work; + stream_type::socket server; + stream_type::acceptor acceptor; + boost::thread io; + std::atomic<bool> connected; + + io_thread() + : io_service(), + work(io_service), + server(io_service), + acceptor(io_service), + io([this] () { try { this->io_service.run(); } catch (const std::exception& e) { MERROR(e.what()); }}), + connected(false) + { + acceptor.open(boost::asio::ip::tcp::v4()); + acceptor.bind(stream_type::endpoint{boost::asio::ip::tcp::v4(), 0}); + acceptor.listen(); + acceptor.async_accept(server, [this] (boost::system::error_code error) { + this->connected = true; + if (error) + throw boost::system::system_error{error}; + }); + } + + ~io_thread() noexcept + { + io_service.stop(); + if (io.joinable()) + io.join(); + } + }; + + struct checked_client + { + std::atomic<bool>* called_; + bool expected_; + + void operator()(boost::system::error_code error, net::socks::client::stream_type::socket&&) const + { + EXPECT_EQ(expected_, bool(error)) << "Socks server: " << error.message(); + ASSERT_TRUE(called_ != nullptr); + (*called_) = true; + } + }; +} + +TEST(socks_client, unsupported_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_connect_command("example.com", 8080)); + EXPECT_TRUE(test_client->buffer().empty()); + + EXPECT_FALSE(test_client->set_resolve_command("example.com")); + EXPECT_TRUE(test_client->buffer().empty()); +} + +TEST(socks_client, no_command) +{ + boost::asio::io_service io_service{}; + stream_type::socket client{io_service}; + + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, std::bind( [] {} ) + ); + ASSERT_TRUE(bool(test_client)); + EXPECT_FALSE(net::socks::client::send(std::move(test_client))); +} + +TEST(socks_client, connect_command) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4a, checked_client{std::addressof(called), false} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_connect_command("example.com", 8080)); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x1f, 0x90, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, connect_command_failed) +{ + io_thread io{}; + stream_type::socket client{io.io_service}; + + std::atomic<bool> called{false}; + auto test_client = net::socks::make_connect_client( + std::move(client), net::socks::version::v4, checked_client{std::addressof(called), true} + ); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE( + test_client->set_connect_command( + epee::net_utils::ipv4_network_address{boost::endian::native_to_big(std::uint32_t(5000)), 3000} + ) + ); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(std::move(test_client), io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 1, 0x0b, 0xb8, 0x00, 0x00, 0x13, 0x88, 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + const std::uint8_t reply_bytes[] = {0, 91, 0, 0, 0, 0, 0, 0}; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (!called); +} + +TEST(socks_client, resolve_command) +{ + static std::uint8_t reply_bytes[] = {0, 90, 0, 0, 0xff, 0, 0xad, 0}; + + struct resolve_client : net::socks::client + { + std::atomic<unsigned> called_; + bool expected_; + + resolve_client(stream_type::socket&& proxy) + : net::socks::client(std::move(proxy), net::socks::version::v4a_tor) + , called_(0) + , expected_(false) + {}; + + virtual void done(boost::system::error_code error, std::shared_ptr<client> self) override + { + EXPECT_EQ(this, self.get()); + EXPECT_EQ(expected_, bool(error)) << "Resolve failure: " << error.message(); + + if (!error) + { + ASSERT_EQ(sizeof(reply_bytes), buffer().size()); + EXPECT_EQ(0u, std::memcmp(buffer().data(), reply_bytes, sizeof(reply_bytes))); + } + + ++called_; + } + }; + + io_thread io{}; + stream_type::socket client{io.io_service}; + + auto test_client = std::make_shared<resolve_client>(std::move(client)); + ASSERT_TRUE(bool(test_client)); + + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::connect_and_send(test_client, io.acceptor.local_endpoint())); + while (!io.connected); + + const std::uint8_t expected_bytes[] = { + 4, 0xf0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, + 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'o', 'm', 0x00 + }; + + std::uint8_t actual_bytes[sizeof(expected_bytes)]; + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 0); + + test_client->expected_ = true; + ASSERT_TRUE(test_client->set_resolve_command("example.com")); + EXPECT_FALSE(test_client->buffer().empty()); + ASSERT_TRUE(net::socks::client::send(test_client)); + + boost::asio::read(io.server, boost::asio::buffer(actual_bytes)); + EXPECT_TRUE(std::memcmp(expected_bytes, actual_bytes, sizeof(actual_bytes)) == 0); + + reply_bytes[1] = 91; + boost::asio::write(io.server, boost::asio::buffer(reply_bytes)); + + // yikes! + while (test_client->called_ == 1); +} + + diff --git a/tests/unit_tests/test_peerlist.cpp b/tests/unit_tests/test_peerlist.cpp index f638c6251..03aa48ea0 100644 --- a/tests/unit_tests/test_peerlist.cpp +++ b/tests/unit_tests/test_peerlist.cpp @@ -37,7 +37,7 @@ TEST(peer_list, peer_list_general) { nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); #define MAKE_IPV4_ADDRESS(a,b,c,d,e) epee::net_utils::ipv4_network_address{MAKE_IP(a,b,c,d),e} #define ADD_GRAY_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple; ple.last_seen=last_seen_;ple.adr = addr_; ple.id = id_;plm.append_with_peer_gray(ple);} #define ADD_WHITE_NODE(addr_, id_, last_seen_) { nodetool::peerlist_entry ple;ple.last_seen=last_seen_; ple.adr = addr_; ple.id = id_;plm.append_with_peer_white(ple);} @@ -77,9 +77,180 @@ TEST(peer_list, merge_peer_lists) //([^ \t]*)\t([^ \t]*):([^ \t]*) \tlast_seen: d(\d+)\.h(\d+)\.m(\d+)\.s(\d+)\n //ADD_NODE_TO_PL("\2", \3, 0x\1, (1353346618 -(\4*60*60*24+\5*60*60+\6*60+\7 )));\n nodetool::peerlist_manager plm; - plm.init(false); + plm.init(nodetool::peerlist_types{}, false); std::vector<nodetool::peerlist_entry> outer_bs; #define ADD_NODE_TO_PL(ip_, port_, id_, timestamp_) { nodetool::peerlist_entry ple; epee::string_tools::get_ip_int32_from_string(ple.adr.ip, ip_); ple.last_seen = timestamp_; ple.adr.port = port_; ple.id = id_;outer_bs.push_back(ple);} +} + +namespace +{ + bool check_empty(nodetool::peerlist_storage& peers, std::initializer_list<epee::net_utils::zone> zones) + { + bool pass = false; + for (const epee::net_utils::zone zone : zones) + { + const nodetool::peerlist_types types{peers.take_zone(zone)}; + EXPECT_TRUE(types.white.empty()); + EXPECT_TRUE(types.gray.empty()); + EXPECT_TRUE(types.anchor.empty()); + pass = (types.white.empty() && types.gray.empty() && types.anchor.empty()); + } + return pass; + } +} + +TEST(peerlist_storage, store) +{ + + using address_type = epee::net_utils::address_type; + using zone = epee::net_utils::zone; + + nodetool::peerlist_storage peers{}; + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + + std::string buffer{}; + { + nodetool::peerlist_types types{}; + types.white.push_back({epee::net_utils::ipv4_network_address{1000, 10}, 44, 55}); + types.white.push_back({net::tor_address::unknown(), 64, 75}); + types.gray.push_back({net::tor_address::unknown(), 99, 88}); + types.gray.push_back({epee::net_utils::ipv4_network_address{2000, 20}, 84, 45}); + types.anchor.push_back({epee::net_utils::ipv4_network_address{999, 654}, 444, 555}); + types.anchor.push_back({net::tor_address::unknown(), 14, 33}); + types.anchor.push_back({net::tor_address::unknown(), 24, 22}); + + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::tor, zone::i2p})); + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + nodetool::peerlist_types types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + { + std::ostringstream stream{}; + EXPECT_TRUE(peers.store(stream, types)); + buffer = stream.str(); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); + + { + std::istringstream stream{buffer}; + boost::optional<nodetool::peerlist_storage> read_peers = + nodetool::peerlist_storage::open(stream, true); + ASSERT_TRUE(bool(read_peers)); + peers = std::move(*read_peers); + } + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::i2p})); + + types = peers.take_zone(zone::public_); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::ipv4, types.white[0].adr.get_type_id()); + EXPECT_EQ(1000u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(10u, types.white[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(44u, types.white[0].id); + EXPECT_EQ(55u, types.white[0].last_seen); + + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::ipv4, types.gray[0].adr.get_type_id()); + EXPECT_EQ(2000u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(20u, types.gray[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(84u, types.gray[0].id); + EXPECT_EQ(45u, types.gray[0].last_seen); + + ASSERT_EQ(1u, types.anchor.size()); + ASSERT_EQ(address_type::ipv4, types.anchor[0].adr.get_type_id()); + EXPECT_EQ(999u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().ip()); + EXPECT_EQ(654u, types.anchor[0].adr.template as<epee::net_utils::ipv4_network_address>().port()); + EXPECT_EQ(444u, types.anchor[0].id); + EXPECT_EQ(555u, types.anchor[0].first_seen); + + types = peers.take_zone(zone::tor); + EXPECT_TRUE(check_empty(peers, {zone::invalid, zone::public_, zone::i2p, zone::tor})); + + ASSERT_EQ(1u, types.white.size()); + ASSERT_EQ(address_type::tor, types.white[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.white[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.white[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(64u, types.white[0].id); + EXPECT_EQ(75u, types.white[0].last_seen); + ASSERT_EQ(1u, types.gray.size()); + ASSERT_EQ(address_type::tor, types.gray[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.gray[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.gray[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(99u, types.gray[0].id); + EXPECT_EQ(88u, types.gray[0].last_seen); + ASSERT_EQ(2u, types.anchor.size()); + ASSERT_EQ(address_type::tor, types.anchor[0].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[0].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[0].adr.template as<net::tor_address>().port()); + EXPECT_EQ(14u, types.anchor[0].id); + EXPECT_EQ(33u, types.anchor[0].first_seen); + ASSERT_EQ(address_type::tor, types.anchor[1].adr.get_type_id()); + EXPECT_STREQ(net::tor_address::unknown_str(), types.anchor[1].adr.template as<net::tor_address>().host_str()); + EXPECT_EQ(0u, types.anchor[1].adr.template as<net::tor_address>().port()); + EXPECT_EQ(24u, types.anchor[1].id); + EXPECT_EQ(22u, types.anchor[1].first_seen); } |