############################### External Network Considerations ############################### .. _networking-external_allocations: Allocations =========== We've got a lot of IP addresses and ranges allocated to us by various parties. Here's an attempt to keep track of them all. +----------------------+-----------------+--------------------------------+--------------+ | Address/block | Where | What | From Whom | +======================+=================+================================+==============+ | 128.220.70.0/24 | Malone VLAN 13 | Cluster JHU Internal ("oldcs") | JHU IT | +----------------------+-----------------+--------------------------------+--------------+ | 128.220.251.32/29 | Malone DMZ | Cluster JHU DMZ ("ff") | JHU IT | +----------------------+-----------------+--------------------------------+--------------+ | 128.220.35.176/28 | Malone VLAN 35 | CS public subnet | CS | +----------------------+-----------------+--------------------------------+--------------+ | 10.161.159.216/29 | Malone VLAN 159 | CS private subnet | CS | +----------------------+-----------------+--------------------------------+--------------+ | 2606:2B00:0:410::/64 | ??? | JHU IPv6 network | JHU IT | +----------------------+-----------------+--------------------------------+--------------+ Internally, the subnets from JHU IT are allocated to: +-------------------+----------------------------+---------------------------------+ | Address/block | Controller | What | +===================+============================+=================================+ | 128.220.70.0/25 | Various | ACM servies and physical hosts | +-------------------+----------------------------+---------------------------------+ | 128.220.70.128/26 | Gomes via Openstack | ACM virtual machines | +-------------------+----------------------------+---------------------------------+ | 128.220.70.192/26 | Gomes via OpenStack | User virtual machines | +-------------------+----------------------------+---------------------------------+ | 128.220.251.32/29 | Magellan, Gomes | See tables below | +-------------------+----------------------------+---------------------------------+ .. note :: As of this writing, 251.38 is unallocated. Security Policies ----------------- Network security manages the JHU border gateway policy for ``128.220.70.0/24`` and requires us to have a clean-slate report to their scanning tools for external access to be granted. Contact network.security@jhu.edu to get the policy adjusted, but please try to keep the tables below up-to-date, too! This means, among other things, that we are obligated to not attempt IP-address-based restrictions that would keep the following IP addresses from probing our systems: ``10.181.169.162``, ``10.181.169.163``, ``10.181.169.164``, ``10.15.69.217``, ``10.15.69.218``, ``10.15.69.219``, ``128.220.242.60``, ``10.131.228.26``, and ``10.132.160.55``. Thankfully, for the most part, public services offered by our cluster are not restricted by IP address anyway. Naming ------ Details of how subnet DNS entries are managed can be found in :ref:`dns_external`. DHCP or other Dynamic Configuration ----------------------------------- Our allocations from CS can be managed by CS's DHCP server; for somewhat obvious reasons they don't want us running our own on their network. To adjust the MAC/IP map, send mail to ``support@cs``. Our direct allocations are manually managed and do not use dynamic configuration. Cluster Common Considerations ============================= The cluster is a big mess internally that gets services exposed on a handful of IP addresses, both inside the JHU firewall and outside and has somewhat interesting egress rules. This page attempts to document the thinking behind some of our port maps, but is *non-authoritative* (the authority, of course, is what is configured on the cluster gateway). For the moment, we use ``shorewall`` to manage our network configuration. .. _enet-egress: Multi-Provider Egress and Tracking ---------------------------------- We have two providers configured, in ``/etc/shorewall/providers``:: csprov 1 0x1000 main $NET_IF_CS 128.220.70.1 track $NET_IFS_INTERNAL ffprov 2 0x2000 main $NET_IF_FF 128.220.251.33 track $NET_IFS_INTERNAL The ``track`` directive ensures that we route responses back out the interface on which things arrived. ``/etc/shorewall/rtrules`` describes the egress rules. This file differs between Magellan and Gomes, but in rough schematic:: #SOURCE DEST PROVIDER PRIORITY - 10.0.0.0/8 csprov 26000 $NET_CIDR_OS_A_CS - csprov 26000 $NET_CIDR_OS_A_FF - ffprov 26000 $NET_CIDR_OS_U_CS - csprov 26000 These rules ensure that, unless otherwise indicated by the ingress-attached tracking labels, that outbound traffic to JHU-internal RFC1918 addresses egress via the behind-firewall interface. The ``$NET_CIDR_OS`` lines dictate how egress from our OpenStack VMs is routed -- the ``_A_`` regions are for VMs under administrative control while ``_U_`` are for VMs running user code. The contents of ``/etc/shorewall/masq`` follow along. Again, this file differs between Magellan and Gomes, but in rough sketch:: $NET_CS_IF $NET_OS_A_CS_CIDR $NET_FF_IF $NET_OS_A_FF_CIDR $NET_CS_IF $NET_OS_U_CS_CIDR $AEOLUS_CS_EXT .. _enet-ingress: Ingress ------- It will probably be clearer to present the contents of ``/etc/shorewall/rules`` in a tabular form: Magellan ```````` +-------------+-----------+------------+-----------------------------------------------+ | IP Address | Port | JHU Public | Description | +=============+===========+============+===============================================+ | .70.63 | | | | | (magellan) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP | No | | | | 22 | | Magellan itself listening on SSH | | +-----------+------------+-----------------------------------------------+ | | UDP | Yes | Alt. address for 128.220.251.36 file server | | | 7000,7005 | | | +-------------+-----------+------------+-----------------------------------------------+ | .70.64 | | | | | (magellan2) | | | | | +-----------+------------+-----------------------------------------------+ | | UDP | Yes | Alt. address for 128.220.251.35 file server | | | 7000,7005 | | | +-------------+-----------+------------+-----------------------------------------------+ | .251.34 | | DMZ | | | (seattle) | | | | | +-----------+------------+-----------------------------------------------+ | | | | (User firewall free egress address: | | | | | :ref:`enet-egress` | | +-----------+------------+-----------------------------------------------+ | | UDP | | | | | 7000,7005 | | AFS scratch and mirror server | +-------------+-----------+------------+-----------------------------------------------+ | .251.36 | | DMZ | | | (magellan) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP | | | | | 22 | | Magellan itself listening on SSH | | +-----------+------------+-----------------------------------------------+ | | UDP | | | | | 7000,7005 | | AFS homedirs and services server | +-------------+-----------+------------+-----------------------------------------------+ Gomes ````` +-------------+-----------+------------+-----------------------------------------------+ | IP Address | Port | JHU Public | Description | +=============+===========+============+===============================================+ | .70.55 | | | | | (astrolabe) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP | Yes | | | | 80, 443 | | Mirrors web server | +-------------+-----------+------------+-----------------------------------------------+ | .70.65 | | | | | (centaur) | | | | | [enet-acm]_ | | | | | +-----------+------------+-----------------------------------------------+ | | TCP | Yes | | | | 22 | | All-users SSH server (conch) | | +-----------+------------+-----------------------------------------------+ | | TCP | Yes | | | | 80, 443 | | User web server (web.vm) | | +-----------+------------+-----------------------------------------------+ | | TCP 25, | Yes | | | | 465, 587 | | ACM mail service (centaur.vm) | +-------------+-----------+------------+-----------------------------------------------+ | .70.74 | | | | | (nagios) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP 80 | Yes | Nagios worker machine (bigbrother.trinidad) | | +-----------+------------+-----------------------------------------------+ | | ICMP | | Nagios worker machine (bigbrother.trinidad) | +-------------+-----------+------------+-----------------------------------------------+ | .70.79 | | | | | | | | | +-------------+-----------+------------+-----------------------------------------------+ | .70.82 | | | | | (belthazar) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP | Yes | | | | 22 | | :doc:`../../services/egg` | + +-----------+------------+-----------------------------------------------+ | | TCP | Yes | | | | 80,443 | | Mailman web interface (lists.acm.jhu.edu) | + +-----------+------------+-----------------------------------------------+ | | | | (User firewalled egress address: | | | | | :ref:`enet-egress` | +-------------+-----------+------------+-----------------------------------------------+ | .70.84 | | | | | | | | | +-------------+-----------+------------+-----------------------------------------------+ | .70.90 | | | | | | | | | +-------------+-----------+------------+-----------------------------------------------+ | .70.91 | | | | | | | | | +-------------+-----------+------------+-----------------------------------------------+ | .251.35 | | DMZ | | | (batman) | | | | | +-----------+------------+-----------------------------------------------+ | | TCP 22 | | All-users SSH server (conch.ff.uvm) | | +-----------+------------+-----------------------------------------------+ | | TCP | | | | | 4242 | | Quassel IRC agent (quassel.vm) | | +-----------+------------+-----------------------------------------------+ | | TCP | | | | | 6080 | | Sandstorm alias | +-------------+-----------+------------+-----------------------------------------------+ | .251.37 | | DMZ | | | (london) | | | | +-------------+-----------+------------+-----------------------------------------------+ .. [enet-acm] For historical reasons, we have an A record in DNS for our domain. This IP address should probably have the "canonically ACM" things listening on it. At present, this address is *inside the JHU firewall*. .. todo:: It might be nice to have this table generated automatically from the contents of the various rules files, actually. No? Services Without the Cluster ============================ For the sake of eliminating SPOFs on critical services, the following services are run on hosts entirely outside the cluster gateway. So even if everything falls over, authentication and name resolution should continue to function. +-------------+-----------+------------+----------------------------------+ | IP Address | Port | JHU Public | Description | +=============+===========+============+==================================+ | .70.76 | | | | | (typhon) | | | | | | | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | SSH | | | 22 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`../auth/ldap` | | | 389 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`../auth/kdc` | | | 88 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS Fileserver (esp. replicas) | | | 7000 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS DBs | | | 7002,7003 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS VolSer | | | 7005 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS BosServer | | | 7007 | | | +-------------+-----------+------------+----------------------------------+ | .70.53 | | | Mail server | | (crimea) | | | | | | | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | SSH | | | 22 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | Mail ingress | | | 25 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | Mailing list web interface | | | 80/443 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS Fileserver reservation | | | 7000 | | Crimea is not an AFS server now! | +-------------+-----------+------------+----------------------------------+ | .35.178 | | | (Services here are replicas from | | (echidna) | | | Typhon) | | | | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | SSH | | | 22 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`../auth/ldap` | | | 389 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`../auth/kdc` | | | 88 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS Fileserver (esp. replicas) | | | 7000 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS DBs | | | 7002,7003 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS VolSer | | | 7005 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS BosServer | | | 7007 | | | +-------------+-----------+------------+----------------------------------+ | .35.191 | | | (Most services here are replicas | | (chicago) | | | from Typhon) | | | | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | SSH | | | 22 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | TCP | Yes | :doc:`../auth/ldap` | | | 389 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`dns` | | | 53 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | :doc:`../auth/kdc` | | | 88 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS Fileserver (esp. replicas) | | | 7000 | | | | +-----------+------------+----------------------------------+ | | UDP | Yes | AFS DBs | | | 7002,7003 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS VolSer | | | 7005 | | | | +-----------+------------+----------------------------------+ | | UDP | No | AFS BosServer | | | 7007 | | | +-------------+-----------+------------+----------------------------------+ .. _networking-external_cluster-uplink-cabling: Cluster Uplink Cabling ====================== You may also wish to refer to :ref:`networking-internal_cluster-switch-cabling` for the inside job. +---------------------------------------+--------------------+ | Host and port | Neighbor | +=======================================+====================+ | Gomes leftmost (eth0) | oldcs (70) | +---------------------------------------+--------------------+ | Magellan leftmost (eth0) | oldcs (70) | +---------------------------------------+--------------------+ | Magellan next leftmost (eth1) | DMZ | +---------------------------------------+--------------------+ | Crimea leftmost | oldcs (70) | +---------------------------------------+--------------------+ | Chicago eth1 | oldcs (70) | +---------------------------------------+--------------------+ | Typhon eth0 | oldcs (70) | +---------------------------------------+--------------------+ | Echidna | cs public (35) | +---------------------------------------+--------------------+