OpenBSD Journal

Prioritizing empty TCP ACKs with pf and ALTQ

Contributed by jose on from the speedier-downloads dept.

Daniel Hartmeier contributes:
"If you're using an ADSL connection, this little article might be useful to you (it certainly is for me). Otherwise it shows some of the new ALTQ syntax in pf.conf, and there are nice graphs, too :)"
Daniel and PF kick butt, that is all. I first found this via a message to the PF list from Daniel on the subject.

(Comments are closed)

Comments

  1. By grey () on

    I have been waiting for this ever since altq got merged in. Glad to hear it's finally possible! I could be wrong (so don't go on my word) but this might be one of the first working workarounds to this issue. I was at a Packeteer sales pitch a while back, and asked if they could offer that kind of functionality (I mean, heck they just do traffic shaping after all, and they charge quite a bit) and didn't get an answer in the affirmative. :-/

    Yet again - pf pulls through. Whenever the hint-dropped failover implementation comes (is 3.4 too hopeful?) I think pf will be ready to wipe the mat.

    Kudos to Henning, Daniel et al - yet again!

    Comments

    1. By Philipp () pb@ on mailto:pb@

      hint-dropped failover..

      think about anchors and tables and your own daemon
      checking if the service is *really* still
      available.. it already works ;)

      an in-kernel check via ping or tracking tcp
      handshakes is only bloat and wont really gain
      anything wrt reliability..

      pb

    2. By Anonymous Coward () on

      Linux has been able to do this for years:
      http://lartc.org/
      http://lartc.org/wondershaper/

      I'm not trolling, I'm just glad this is going to be in 3.3. I really didn't want to switch back to using linux on my firewall/router.

      Comments

      1. By Anonymous Coward () on

        Before 3.3 you could use ALTQ, but now it's just integrated with PF.

        Comments

        1. By Anonymous Coward () on

          right. But that prevented using techniques such as that one described in this article.

  2. By matt ostiguy () matt at ostiguy.com on mailto:matt at ostiguy.com

    he is approaching schneier-esque heights in being both a great technologist, as well as a great writer.

  3. By Anonymous Coward () on

    Am i happy with this? yeah! i had a script for this with QoS @ Linux but my home box doesn't run Linux. Yay now i can do this with OpenBSD!

    For educational purpose i'll post the script. Author unknown. Here it is:

    #!/usr/local/bin/bash

    # The Ultimate Setup For Your Internet Connection At Home
    #
    #
    # Set the following values to somewhat less than your actual download
    # and uplink speed. In kilobits
    # (EDIT THESE)
    DOWNLINK=1024
    UPLINK=256
    DEV=random

    # clean existing down- and uplink qdiscs, hide errors
    tc qdisc del dev $DEV root 2> /dev/null > /dev/null
    tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null

    ###### uplink

    # install root CBQ

    tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 10mbit

    # shape everything at $UPLINK speed - this prevents huge queues in your
    # DSL modem which destroy latency:
    # main class

    tc class add dev $DEV parent 1: classid 1:1 cbq rate ${UPLINK}kbit
    allot 1500 prio 5 bounded isolated

    # high prio class 1:10:

    tc class add dev $DEV parent 1:1 classid 1:10 cbq rate ${UPLINK}kbit
    allot 1600 prio 1 avpkt 1000

    # bulk and default class 1:20 - gets slightly less traffic,
    # and a lower priority:

    tc class add dev $DEV parent 1:1 classid 1:20 cbq rate $[9*$UPLINK/10]kbit
    allot 1600 prio 2 avpkt 1000

    # both get Stochastic Fairness:
    tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
    tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10

    # start filters
    # TOS Minimum Delay (ssh, NOT scp) in 1:10:
    tc filter add dev $DEV parent 1:0 protocol ip prio 10 u32
    match ip tos 0x10 0xff flowid 1:10

    # ICMP (ip protocol 1) in the interactive class 1:10 so we
    # can do measurements & impress our friends:
    tc filter add dev $DEV parent 1:0 protocol ip prio 11 u32
    match ip protocol 1 0xff flowid 1:10

    # To speed up downloads while an upload is going on, put ACK packets in
    # the interactive class:

    tc filter add dev $DEV parent 1: protocol ip prio 12 u32
    match ip protocol 6 0xff
    match u8 0x05 0x0f at 0
    match u16 0x0000 0xffc0 at 2
    match u8 0x10 0xff at 33
    flowid 1:10

    # rest is 'non-interactive' ie 'bulk' and ends up in 1:20

    tc filter add dev $DEV parent 1: protocol ip prio 13 u32
    match ip dst 0.0.0.0/0 flowid 1:20

    ########## downlink #############
    # slow downloads down to somewhat less than the real speed to prevent
    # queuing at our ISP. Tune to see how high you can set it.
    # ISPs tend to have *huge* queues to make sure big downloads are fast
    #
    # attach ingress policer:

    tc qdisc add dev $DEV handle ffff: ingress

    # filter *everything* to it (0.0.0.0/0), drop everything that's
    # coming in too fast:

    tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src
    0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1

    Comments

    1. By Anonymous Coward () on

      Neat. The backslashes 's got ripped of

    2. By Anonymous Coward () on

      drop everything that's coming in too fast...

      You are real QoS expert (tm)

      Comments

      1. By Anonymous Coward () on

        1) we are talking about ADSL connections here hun.
        2) read the fucking comments (tm)

        Comments

        1. By Anonymous Coward () on

          OK Read TFM, Got it !!! # everything that's coming in too fast: is actually a command, denoted by hash mark and tc filter add dev $DEV parent ffff: protocol Is a comment, got it, no more questions

          Comments

          1. By Anonymous Coward () on

            Do you still remember to breath into your mouth and out of your nose?

            Comments

            1. By Troll () on

              Sure thing

    3. By ac () on

      haven`t checked, but i think it`s the popular wondershaper:

      http://lartc.org/wondershaper/

  4. By Anonymous Coward () on

    Most disciplines of altq has gone, where is the catch ???

    Comments

    1. By Henning () henning@cvs.openbsd.org on mailto:henning@cvs.openbsd.org

      look at where altq comes from. a research project. most of the queueing diciplines were purely research orientate with little to no real world relevance. so we ditched everything besides cbq and priq, and I will add hfsc back after 3.3. as said, we do not believe the other schedulers have any real world relevance.

      Comments

      1. By IIorT () on

        Somehow over the years i learned to discipline traffic using altq alone (no pf) as i manage both ends of (nearly saturated) links. So I imagined ALTQ will get hinted from PF connection tracking, and do the additional tricks with slowing acks and sending ICMP source quenches to juggle input streams (like kind http proxies and ftp clients and servers do) ... not that i needed it, but that coud be interesting.

      2. By Anonymous Coward () on

        Its an easy way to let a bunch of users share a link without anyone being able to hog it all. What do you propose people use instead?

        Comments

        1. By IIorT () on

          "ALTQ is integrated into KAME IPv6 and being developed under the KAME CVS repository."

  5. By Andy () on

    I'm confused... I thought that altq could only work outbound on an interface, yet the rules here indicate that it is also working on inbound. Is the ``pass in on...'' rule there just for looks?

    Comments

    1. By Daniel Hartmeier () daniel@benzedrine.cx on http://www.benzedrine.cx/pf.html

      Note the 'keep state' on that 'pass in' rule. When an incoming connection creates a state, the queues to assign packets to are stored in the state entry, and further _outgoing_ packets related to the connection are assigned to these queues.

      So the reason for the 'pass in ... keep state queue (...)' rule is to enqueue the outgoing replies for incoming connections.

      Comments

      1. By Robert Mooney () on

        Daniel, what program did you use to generate the graphs?

        Comments

        1. By Daniel Hartmeier () daniel@benzedrine.cx on http://www.benzedrine.cx/pf.html

          pfstat, see http://www.benzedrine.cx/pfstat.html
          In -current, there's a port under ports/net/pfstat.

    2. By Henning () henning@cvs.openbsd.org on mailto:henning@cvs.openbsd.org

      altq works only outbound.
      HOWEVER... in the typical case where you are forwarding, inbound on $ext_if means outbound on $int_if, and you can do the assignment to queues living on $int_if with rules on $ext_if.
      also keep in mind we are stateful.
      thus,
      pass in on $ext_if keep state queue blah
      might not have effect on the incoming packets, including and expecially the one(s) that create the state, but on the outbound packets matching the state entry created from this rule.

  6. By AC () on

    How to do the same in OpenBSD 3.2?

    Comments

    1. By Sam () on

      Install a snapshot and test it.

    2. By Anonymous Coward () on

      /etc/altq.conf:
      interface bandwidth K wfq

      needs kernel rebuild (GENERIC will not do)

      a bit different but works most of the time ie - automagically assigns ack queue a fair share of resources and slows tcp data output to reasonable speed to efficiently use all of data channel

    3. By Anonymous Coward () on

      i mean:

      interface (interface) bandwidth (subscribed)K wfq

    4. By Anonymous Coward () on

      Despite what the above poster said, you cannot do this with 3.2. You will be able to in 3.3 (or in a current snapshot) when ALTQ and PF are merged.

      Yes, you can do other traffic shaping now, but not what you want.

      if you want to know more about what you can do in 3.2, run "man altq".

    5. By Henning () henning@cvs.openbsd.org on mailto:henning@cvs.openbsd.org

      well, pick your favorite ftp mirror, get bsd and all teh .tgzs, and upgrade to 3.3-beta ;-)

  7. By Anonymous Coward () on

    A quick comparison between the -current pf.conf manpage and the one on my 3.2 box shows quite some interesting improvements.
    It seems pf has gotten way better than I would've ever dared to dream :)
    Can't hardly wait for 3.3 :)

    Comments

    1. By Jason Dixon () jwdixon1@yahoo.com on http://www.jasondixon.net

      Download the latest snapshot and try it out! I'm using a recent snapshot now, works great. Wait until the sendmail patch has been merged into snapshot, and you'll be set. :)

      -J.

  8. By thebiMbo () on

    Daniel what do you think about dummynet in the ipfw toolset ?
    Have you evere looked at that one !?

    Regards

  9. By Mark () mark@hornclan.com on mailto:mark@hornclan.com

    I'm a linux user. I am, however, curious about OpenBSD. After reading the article, and doing a little bit of testing, it seems like my Linux device handles this by default on my asymetric cable modem (2.0MB down, 256k up).

    Can someone describe a test for me that would demonstrate the problem? I've tried uploading a large file while downloading another large file, and been able to max the bandwidth in both directions.

    What am I missing?

    Comments

    1. By Anonymous Coward () on

      basically you can do this by simultaneously uploading and downloading and attempting to connect to ssh, ftp or another interactive service from outside.
      there is nothing by default.
      some providers do queuing and you will not notice any problem whatsoever.
      you need at least two downloads or two uploads to spot difference.

  10. By jakennedy () on

    Is there a reason why it is not appropriate for the kernel to be doing this prioritization? It seems like the appropriate place for such a logical (and elegant) solution.

    Comments

    1. By Henning () henning@cvs.openbsd.org on mailto:henning@cvs.openbsd.org

      of course it is the kernel who does that. with pfctl, you just feed the kernel the information how to do that.

      Comments

      1. By jakennedy () on

        I guess a better question would be, is it not appropriate to be doing this by default?

  11. By jakennedy () on

    Is there a reason why it is not appropriate for the kernel to be doing this prioritization? It seems like the appropriate place for such a logical (and elegant) solution.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]