Change fee escalation algorithms (RIPD-1177):

* Minimum factor 256*500, don't multiply by base fee
* Change autofill fee behavior to pay the open ledger fee.
** Experimental options: x-assume-tx - assume <int> more transactions in
   the open queue when computing escalated fee, x-queue-okay - if true
   and escalated fee is over limit, try with load fee.

src/ripple/app/misc/FeeEscalation.md

@@ -35,30 +35,30 @@ consensus process, but will be at least [5](#other-constants).
   or the number of transactions in the validated ledger.
 3. Once there are more transactions in the open ledger than indicated
 by the limit, the required fee level jumps drastically.
-  * The formula is `( baseFeeLevel * lastLedgerMedianFeeLevel *
+  * The formula is `( lastLedgerMedianFeeLevel *
   TransactionsInOpenLedger^2 / limit^2 )`,
   and returns a [fee level](#fee-level).
 4. That may still be pretty small, but as more transactions get
 into the ledger, the fee level increases exponentially.
   * For example, if the limit is 6, and the median fee is minimal,
   and assuming all [reference transactions](#reference-transaction),
-  the 7th transaction only requires a [level](#fee-level) of about 174,000
+  the 8th transaction only requires a [level](#fee-level) of about 174,000
   or about 6800 drops,
   but the 20th transaction requires a [level](#fee-level) of about
-  1,422,000 or about 56,000 drops.
+  1,283,000 or about 50,000 drops.
 5. Finally, as each ledger closes, the median fee level of that ledger is
 computed and used as `lastLedgerMedianFeeLevel` (with a
-[minimum value of 500](#other-constants))
+[minimum value of 128,000](#other-constants))
 in the fee escalation formula for the next open ledger.
   * Continuing the example above, if ledger consensus completes with
   only those 20 transactions, and all of those transactions paid the
   minimum required fee at each step, the limit will be adjusted from
-  6 to 20, and the `lastLedgerMedianFeeLevel` will be about 393,000,
+  6 to 20, and the `lastLedgerMedianFeeLevel` will be about 322,000,
-  which is 15,000 drops for a
+  which is 12,600 drops for a
   [reference transaction](#reference-transaction).
   * This will cause the first 21 transactions only require 10
-  drops, but the 22st transaction will require
+  drops, but the 22nd transaction will require
-  a level of about 110,000,000 or about 4.3 million drops (4.3XRP).
+  a level of about 355,000 or about 13,800 drops.
 
 ## Transaction Queue
 
@@ -71,13 +71,14 @@ allows legitimate users to continue submitting transactions during high
 traffic periods, and give those transactions a much better chance to
 succeed.
 
-1. If an incoming transaction meets the base [fee level](#fee-level),
+1. If an incoming transaction meets both the base [fee
-but does not have a high enough [fee level](#fee-level) to immediately
+level](#fee-level) and the load fee minimum, but does not have a high
-go into the open ledger, it is instead put into the queue and broadcast
+enough [fee level](#fee-level) to immediately go into the open ledger,
-to peers. Each peer will then make an independent decision about whether
+it is instead put into the queue and broadcast to peers. Each peer will
-to put the transaction into its open ledger or the queue. In principle,
+then make an independent decision about whether to put the transaction
-peers with identical open ledgers will come to identical decisions. Any
+into its open ledger or the queue. In principle, peers with identical
-discrepancies will be resolved as usual during consensus.
+open ledgers will come to identical decisions. Any discrepancies will be
+resolved as usual during consensus.
 2. When consensus completes, the open ledger limit is adjusted, and
 the required [fee level](#fee-level) drops back to the base
 [fee level](#fee-level). Before the ledger is made available to
@@ -93,12 +94,11 @@ but in practice, either
   * it will eventually get applied to the ledger,
   * its last ledger sequence number will expire,
   * the user will replace it by submitting another transaction with the same
-  sequence number and a higher fee, or
+  sequence number and a [25% higher fee](#other-constants), or
   * it will get dropped when the queue fills up with more valuable transactions.
   The size limit is computed dynamically, and can hold transactions for
-  the next [20 ledgers](#other-constants).
+  the next [20 ledgers](#other-constants). The lower the transaction's
-  The lower the transaction's fee, the more likely that it will get dropped if the
+  fee, the more likely that it will get dropped if the network is busy.
-  network is busy.
 
 Currently, there is an additional restriction that the queue can only hold one
 transaction per account at a time. Future development will make the queue
@@ -156,7 +156,7 @@ unusable. The "target" value of 50 was chosen so the limit never gets large
 enough to invite abuse, but keeps up if the network stays healthy and
 active. These exact values were chosen experimentally, and can easily
 change in the future.
-* *Minimum `lastLedgerMedianFeeLevel`*. The value of 500 was chosen to
+* *Minimum `lastLedgerMedianFeeLevel`*. The value of 128,000 was chosen to
 ensure that the first escalated fee was more significant and noticable
 than what the default would allow. This exact value was chosen
 experimentally, and can easily change in the future.
@@ -168,6 +168,12 @@ to process successfully.  The limit of 20 ledgers was used to provide
 a balance between resource (specifically memory) usage, and giving
 transactions a realistic chance to be processed. This exact value was
 chosen experimentally, and can easily change in the future.
+* *Replaced transaction fee increase*. Any transaction in the queue can be
+replaced by another transaction with the same sequence number and at
+least a 25% higher fee level. The 25% increase is intended to cover the
+resource cost incurred by broadcasting the original transaction to the
+network. This value was chosen experimentally, and can easily change in
+the future.
 
 ### `fee` command
 
@@ -197,26 +203,15 @@ Result format:
          "reference_level" : "256", // level of a reference transaction. Always 256.
          "minimum_level" : "256", // minimum fee level to get into the queue. If >256, indicates the queue is full.
          "median_level" : "281600", // lastLedgerMedianFeeLevel used in escalation calculations.
-         "open_ledger_level" : "82021944" // minimum fee level to get into the open ledger immediately.
+         "open_ledger_level" : "320398" // minimum fee level to get into the open ledger immediately.
       },
       "drops" : {
          "base_fee" : "10", // base fee of a reference transaction in drops.
          "minimum_fee" : "10", // minimum drops to get a reference transaction into the queue. If >base_fee, indicates the queue is full.
          "median_fee" : "11000", // drop equivalent of "median_level" for a reference transaction.
-         "open_ledger_fee" : "3203982" // minimum drops to get a reference transaction into the open ledger immediately.
+         "open_ledger_fee" : "12516" // minimum drops to get a reference transaction into the open ledger immediately.
       }
    }
 }
 ```
 
-### Enabling Fee Escalation
-
-These features are disabled by default and need to be activated by a
-feature in your rippled.cfg. Add a `[features]` section if one is not
-already present, and add `FeeEscalation` (case-sensitive) to that
-list, then restart rippled.
-
-```
-[features]
-FeeEscalation
-```

src/ripple/app/misc/TxQ.h

@@ -66,7 +66,7 @@ public:
         : targetTxnCount_(50)
         , minimumTxnCount_(standAlone ? 1000 : 5)
         , txnsExpected_(minimumTxnCount_)
-        , minimumMultiplier_(500)
+        , minimumMultiplier_(baseLevel * 500)
         , escalationMultiplier_(minimumMultiplier_)
         , j_(j)
     {
@@ -114,7 +114,7 @@ public:
     }
 
     std::uint64_t
-    scaleFeeLevel(OpenView const& view) const;
+    scaleFeeLevel(OpenView const& view, std::uint32_t txCountPadding = 0) const;
 };
 
 }
@@ -242,7 +242,7 @@ public:
     /** Returns fee metrics in reference fee (level) units.
     */
     struct Metrics
-    getMetrics(OpenView const& view) const;
+    getMetrics(OpenView const& view, std::uint32_t txCountPadding = 0) const;
 
     /** Packages up fee metrics for the `fee` RPC command.
     */

src/ripple/app/misc/impl/TxQ.cpp

@@ -149,12 +149,10 @@ FeeMetrics::updateFeeMetrics(Application& app,
 }
 
 std::uint64_t
-FeeMetrics::scaleFeeLevel(OpenView const& view) const
+FeeMetrics::scaleFeeLevel(OpenView const& view, std::uint32_t txCountPadding) const
 {
-    auto fee = baseLevel;
-
     // Transactions in the open ledger so far
-    auto const current = view.txCount();
+    auto const current = view.txCount() + txCountPadding;
 
     std::size_t target;
     std::uint32_t multiplier;
@@ -172,11 +170,10 @@ FeeMetrics::scaleFeeLevel(OpenView const& view) const
     {
         // Compute escalated fee level
         // Don't care about the overflow flag
-        fee = mulDiv(fee, current * current *
+        return mulDiv(multiplier, current * current,
-            multiplier, target * target).second;
+            target * target).second;
     }
-
+    return baseLevel;
-    return fee;
 }
 
 } // detail
@@ -652,7 +649,7 @@ TxQ::accept(Application& app,
 }
 
 TxQ::Metrics
-TxQ::getMetrics(OpenView const& view) const
+TxQ::getMetrics(OpenView const& view, std::uint32_t txCountPadding) const
 {
     Metrics result;
 
@@ -666,7 +663,7 @@ TxQ::getMetrics(OpenView const& view) const
     result.minFeeLevel = isFull() ? byFee_.rbegin()->feeLevel + 1 :
         feeMetrics_.baseLevel;
     result.medFeeLevel = feeMetrics_.getEscalationMultiplier();
-    result.expFeeLevel = feeMetrics_.scaleFeeLevel(view);
+    result.expFeeLevel = feeMetrics_.scaleFeeLevel(view, txCountPadding);
 
     return result;
 }
@@ -707,9 +704,14 @@ TxQ::doRPC(Application& app) const
     drops[jss::median_fee] = to_string(mulDiv(
         metrics.medFeeLevel, baseFee,
             metrics.referenceFeeLevel).second);
-    drops[jss::open_ledger_fee] = to_string(mulDiv(
+    auto escalatedFee = mulDiv(
         metrics.expFeeLevel, baseFee,
-            metrics.referenceFeeLevel).second);
+            metrics.referenceFeeLevel).second;
+    if (mulDiv(escalatedFee, metrics.referenceFeeLevel,
+            baseFee).second < metrics.expFeeLevel)
+        ++escalatedFee;
+
+    drops[jss::open_ledger_fee] = to_string(escalatedFee);
 
     return ret;
 }

src/ripple/app/tests/TxQ_test.cpp

@@ -34,6 +34,8 @@ namespace test {
 
 class TxQ_test : public beast::unit_test::suite
 {
+    static auto constexpr defaultMedianLevel = 256 * 500;
+
     void
     checkMetrics(
         jtx::Env& env,
@@ -53,10 +55,9 @@ class TxQ_test : public beast::unit_test::suite
         expect(metrics.minFeeLevel == expectedMinFeeLevel, "minFeeLevel");
         expect(metrics.medFeeLevel == expectedMedFeeLevel, "medFeeLevel");
         auto expectedCurFeeLevel = expectedInLedger > expectedPerLedger ?
-            metrics.referenceFeeLevel * expectedMedFeeLevel *
+            expectedMedFeeLevel * expectedInLedger * expectedInLedger /
-                expectedInLedger * expectedInLedger /
+                (expectedPerLedger * expectedPerLedger) :
-                    (expectedPerLedger * expectedPerLedger) :
+                    metrics.referenceFeeLevel;
-                        metrics.referenceFeeLevel;
         expect(metrics.expFeeLevel == expectedCurFeeLevel, "expFeeLevel");
     }
 
@@ -98,15 +99,15 @@ public:
 
         expect(env.current()->fees().base == 10);
 
-        checkMetrics(env, 0, boost::none, 0, 3, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 3, 256, defaultMedianLevel);
 
         // Create several accounts while the fee is cheap so they all apply.
         env.fund(XRP(50000), noripple(alice, bob, charlie, daria));
-        checkMetrics(env, 0, boost::none, 4, 3, 256, 500);
+        checkMetrics(env, 0, boost::none, 4, 3, 256, defaultMedianLevel);
 
         // Alice - price starts exploding: held
         env(noop(alice), queued);
-        checkMetrics(env, 1, boost::none, 4, 3, 256, 500);
+        checkMetrics(env, 1, boost::none, 4, 3, 256, defaultMedianLevel);
 
         auto openLedgerFee =
             [&]()
@@ -116,26 +117,25 @@ public:
 
         // Bob with really high fee - applies
         env(noop(bob), openLedgerFee());
-        checkMetrics(env, 1, boost::none, 5, 3, 256, 500);
+        checkMetrics(env, 1, boost::none, 5, 3, 256, defaultMedianLevel);
 
         // Daria with low fee: hold
         env(noop(daria), fee(1000), queued);
-        checkMetrics(env, 2, boost::none, 5, 3, 256, 500);
+        checkMetrics(env, 2, boost::none, 5, 3, 256, defaultMedianLevel);
 
         env.close();
         // Verify that the held transactions got applied
-        auto lastMedian = 500;
+        checkMetrics(env, 0, 10, 2, 5, 256, defaultMedianLevel);
-        checkMetrics(env, 0, 10, 2, 5, 256, lastMedian);
 
         //////////////////////////////////////////////////////////////
 
         // Make some more accounts. We'll need them later to abuse the queue.
         env.fund(XRP(50000), noripple(elmo, fred, gwen, hank));
-        checkMetrics(env, 0, 10, 6, 5, 256, lastMedian);
+        checkMetrics(env, 0, 10, 6, 5, 256, defaultMedianLevel);
 
         // Now get a bunch of transactions held.
         env(noop(alice), fee(12), queued);
-        checkMetrics(env, 1, 10, 6, 5, 256, lastMedian);
+        checkMetrics(env, 1, 10, 6, 5, 256, defaultMedianLevel);
 
         env(noop(bob), fee(10), queued); // won't clear the queue
         env(noop(charlie), fee(20), queued);
@@ -144,12 +144,11 @@ public:
         env(noop(fred), fee(19), queued);
         env(noop(gwen), fee(16), queued);
         env(noop(hank), fee(18), queued);
-        checkMetrics(env, 8, 10, 6, 5, 256, lastMedian);
+        checkMetrics(env, 8, 10, 6, 5, 256, defaultMedianLevel);
 
         env.close();
         // Verify that the held transactions got applied
-        lastMedian = 500;
+        checkMetrics(env, 1, 12, 7, 6, 256, defaultMedianLevel);
-        checkMetrics(env, 1, 12, 7, 6, 256, lastMedian);
 
         // Bob's transaction is still stuck in the queue.
 
@@ -158,59 +157,56 @@ public:
         // Hank sends another txn
         env(noop(hank), fee(10), queued);
         // But he's not going to leave it in the queue
-        checkMetrics(env, 2, 12, 7, 6, 256, lastMedian);
+        checkMetrics(env, 2, 12, 7, 6, 256, defaultMedianLevel);
 
         // Hank sees his txn  got held and bumps the fee,
         // but doesn't even bump it enough to requeue
         env(noop(hank), fee(11), ter(telINSUF_FEE_P));
-        checkMetrics(env, 2, 12, 7, 6, 256, lastMedian);
+        checkMetrics(env, 2, 12, 7, 6, 256, defaultMedianLevel);
 
         // Hank sees his txn got held and bumps the fee,
         // enough to requeue, but doesn't bump it enough to
         // apply to the ledger
         env(noop(hank), fee(6000), queued);
         // But he's not going to leave it in the queue
-        checkMetrics(env, 2, 12, 7, 6, 256, lastMedian);
+        checkMetrics(env, 2, 12, 7, 6, 256, defaultMedianLevel);
 
         // Hank sees his txn got held and bumps the fee,
         // high enough to get into the open ledger, because
         // he doesn't want to wait.
         env(noop(hank), openLedgerFee());
-        checkMetrics(env, 1, 12, 8, 6, 256, lastMedian);
+        checkMetrics(env, 1, 12, 8, 6, 256, defaultMedianLevel);
 
         // Hank then sends another, less important txn
         // (In addition to the metrics, this will verify that
         //  the original txn got removed.)
         env(noop(hank), fee(6000), queued);
-        checkMetrics(env, 2, 12, 8, 6, 256, lastMedian);
+        checkMetrics(env, 2, 12, 8, 6, 256, defaultMedianLevel);
 
         env.close();
 
         // Verify that bob and hank's txns were applied
-        lastMedian = 500;
+        checkMetrics(env, 0, 16, 2, 8, 256, defaultMedianLevel);
-        checkMetrics(env, 0, 16, 2, 8, 256, lastMedian);
 
         // Close again with a simulated time leap to
         // reset the escalation limit down to minimum
-        lastMedian = 76928;
         env.close(env.now() + 5s, 10000ms);
-        checkMetrics(env, 0, 16, 0, 3, 256, lastMedian);
+        checkMetrics(env, 0, 16, 0, 3, 256, defaultMedianLevel);
         // Then close once more without the time leap
         // to reset the queue maxsize down to minimum
-        lastMedian = 500;
         env.close();
-        checkMetrics(env, 0, 6, 0, 3, 256, lastMedian);
+        checkMetrics(env, 0, 6, 0, 3, 256, defaultMedianLevel);
 
         //////////////////////////////////////////////////////////////
 
         // At this point, the queue should have a limit of 6.
         // Stuff the ledger and queue so we can verify that
         // stuff gets kicked out.
-        env(noop(hank));
+        env(noop(hank), fee(7000));
-        env(noop(gwen));
+        env(noop(gwen), fee(7000));
-        env(noop(fred));
+        env(noop(fred), fee(7000));
-        env(noop(elmo));
+        env(noop(elmo), fee(7000));
-        checkMetrics(env, 0, 6, 4, 3, 256, lastMedian);
+        checkMetrics(env, 0, 6, 4, 3, 256, defaultMedianLevel);
 
         // Use explicit fees so we can control which txn
         // will get dropped
@@ -224,7 +220,7 @@ public:
         env(noop(daria), fee(15), queued);
 
         // Queue is full now.
-        checkMetrics(env, 6, 6, 4, 3, 385, lastMedian);
+        checkMetrics(env, 6, 6, 4, 3, 385, defaultMedianLevel);
 
         // Try to add another transaction with the default (low) fee,
         // it should fail because the queue is full.
@@ -236,19 +232,17 @@ public:
         env(noop(charlie), fee(100), queued);
 
         // Queue is still full, of course, but the min fee has gone up
-        checkMetrics(env, 6, 6, 4, 3, 410, lastMedian);
+        checkMetrics(env, 6, 6, 4, 3, 410, defaultMedianLevel);
 
         // Close out the ledger, the transactions are accepted, the
         // queue is cleared, then the localTxs are retried. At this
         // point, daria's transaction that was dropped from the queue
         // is put back in. Neat.
         env.close();
-        lastMedian = 500;
+        checkMetrics(env, 2, 8, 5, 4, 256, 256 * 700);
-        checkMetrics(env, 2, 8, 5, 4, 256, lastMedian);
 
-        lastMedian = 500;
         env.close();
-        checkMetrics(env, 0, 10, 2, 5, 256, lastMedian);
+        checkMetrics(env, 0, 10, 2, 5, 256, defaultMedianLevel);
 
         //////////////////////////////////////////////////////////////
         // Cleanup:
@@ -274,7 +268,7 @@ public:
         checkMetrics(env, metrics.txCount,
             metrics.txQMaxSize, metrics.txPerLedger + 1,
             metrics.txPerLedger,
-            256, lastMedian);
+            256, defaultMedianLevel);
     }
 
     void testLocalTxRetry()
@@ -295,20 +289,20 @@ public:
 
         expect(env.current()->fees().base == 10);
 
-        checkMetrics(env, 0, boost::none, 0, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 2, 256, defaultMedianLevel);
 
         // Create several accounts while the fee is cheap so they all apply.
         env.fund(XRP(50000), noripple(alice, bob, charlie));
-        checkMetrics(env, 0, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Alice - price starts exploding: held
         env(noop(alice), queued);
-        checkMetrics(env, 1, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Alice - Alice is already in the queue, so can't hold.
         env(noop(alice), seq(env.seq(alice) + 1),
             ter(telINSUF_FEE_P));
-        checkMetrics(env, 1, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 3, 2, 256, defaultMedianLevel);
 
         auto openLedgerFee =
             [&]()
@@ -319,23 +313,22 @@ public:
         // fails because the item in the TxQ hasn't applied.
         env(noop(alice), openLedgerFee(),
             seq(env.seq(alice) + 1), ter(terPRE_SEQ));
-        checkMetrics(env, 1, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Bob with really high fee - applies
         env(noop(bob), openLedgerFee());
-        checkMetrics(env, 1, boost::none, 4, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 4, 2, 256, defaultMedianLevel);
 
         // Daria with low fee: hold
         env(noop(charlie), fee(1000), queued);
-        checkMetrics(env, 2, boost::none, 4, 2, 256, 500);
+        checkMetrics(env, 2, boost::none, 4, 2, 256, defaultMedianLevel);
 
         env.close();
         // Verify that the held transactions got applied
-        auto lastMedian = 500;
         // One of alice's bad transactions applied from the
         // Local Txs. Since they both have the same seq,
         // one succeeds, one fails. We don't care which.
-        checkMetrics(env, 0, 8, 3, 4, 256, lastMedian);
+        checkMetrics(env, 0, 8, 3, 4, 256, defaultMedianLevel);
     }
 
     void testLastLedgerSeq()
@@ -357,7 +350,7 @@ public:
 
         auto queued = ter(terQUEUED);
 
-        checkMetrics(env, 0, boost::none, 0, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 2, 256, defaultMedianLevel);
 
         // Fund across several ledgers so the TxQ metrics stay restricted.
         env.fund(XRP(1000), noripple(alice, bob));
@@ -367,25 +360,25 @@ public:
         env.fund(XRP(1000), noripple(edgar, felicia));
         env.close(env.now() + 5s, 10000ms);
 
-        checkMetrics(env, 0, boost::none, 0, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 2, 256, defaultMedianLevel);
         env(noop(bob));
         env(noop(charlie));
         env(noop(daria));
-        checkMetrics(env, 0, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Queue an item with a LastLedgerSeq.
         env(noop(alice), json(R"({"LastLedgerSequence":7})"),
             queued);
         // Queue items with higher fees to force the previous
         // txn to wait.
-        env(noop(bob), fee(20), queued);
+        env(noop(bob), fee(7000), queued);
-        env(noop(charlie), fee(20), queued);
+        env(noop(charlie), fee(7000), queued);
-        env(noop(daria), fee(20), queued);
+        env(noop(daria), fee(7000), queued);
-        env(noop(edgar), fee(20), queued);
+        env(noop(edgar), fee(7000), queued);
-        checkMetrics(env, 5, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 5, boost::none, 3, 2, 256, defaultMedianLevel);
 
         env.close();
-        checkMetrics(env, 1, 6, 4, 3, 256, 500);
+        checkMetrics(env, 1, 6, 4, 3, 256, defaultMedianLevel);
 
         // Keep alice's transaction waiting.
         env(noop(bob), fee(20), queued);
@@ -393,12 +386,12 @@ public:
         env(noop(daria), fee(20), queued);
         env(noop(edgar), fee(20), queued);
         env(noop(felicia), fee(20), queued);
-        checkMetrics(env, 6, 6, 4, 3, 257, 500);
+        checkMetrics(env, 6, 6, 4, 3, 257, defaultMedianLevel);
 
         env.close();
         // alice's transaction expired without getting
         // into the ledger, so the queue is now empty.
-        checkMetrics(env, 0, 8, 5, 4, 256, 512);
+        checkMetrics(env, 0, 8, 5, 4, 256, 179200);
         expect(env.seq(alice) == 1);
     }
 
@@ -417,7 +410,7 @@ public:
 
         auto queued = ter(terQUEUED);
 
-        checkMetrics(env, 0, boost::none, 0, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 2, 256, defaultMedianLevel);
 
         // Fund these accounts and close the ledger without
         // involving the queue, so that stats aren't affected.
@@ -428,17 +421,17 @@ public:
         env(noop(alice));
         env(noop(alice));
         env(noop(alice));
-        checkMetrics(env, 0, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 3, 2, 256, defaultMedianLevel);
 
         env(noop(bob), queued);
-        checkMetrics(env, 1, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Even though this transaction has a 0 fee,
         // SetRegularKey::calculateBaseFee indicates this is
         // a "free" transaction, so it has an "infinite" fee
         // level and goes into the open ledger.
         env(regkey(alice, bob), fee(0));
-        checkMetrics(env, 1, boost::none, 4, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 4, 2, 256, defaultMedianLevel);
 
         // This transaction also has an "infinite" fee level,
         // but since bob has a txn in the queue, and multiple
@@ -448,7 +441,7 @@ public:
         // canBeHeld failing under the hood.
         env(regkey(bob, alice), fee(0),
             seq(env.seq(bob) + 1), ter(terPRE_SEQ));
-        checkMetrics(env, 1, boost::none, 4, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 4, 2, 256, defaultMedianLevel);
 
     }
 
@@ -490,19 +483,19 @@ public:
 
         auto queued = ter(terQUEUED);
 
-        checkMetrics(env, 0, boost::none, 0, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 0, 2, 256, defaultMedianLevel);
 
         env.fund(XRP(1000), noripple(alice, bob));
 
-        checkMetrics(env, 0, boost::none, 2, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 2, 2, 256, defaultMedianLevel);
 
         // Fill the ledger
         env(noop(alice));
-        checkMetrics(env, 0, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 0, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Put a transaction in the queue
         env(noop(alice), queued);
-        checkMetrics(env, 1, boost::none, 3, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 3, 2, 256, defaultMedianLevel);
 
         // Now cheat, and bypass the queue.
         {
@@ -524,12 +517,12 @@ public:
                 );
             env.postconditions(jt, ter, didApply);
         }
-        checkMetrics(env, 1, boost::none, 4, 2, 256, 500);
+        checkMetrics(env, 1, boost::none, 4, 2, 256, defaultMedianLevel);
 
         env.close();
         // Alice's queued transaction failed in TxQ::accept
         // with tefPAST_SEQ
-        checkMetrics(env, 0, 8, 0, 4, 256, 500);
+        checkMetrics(env, 0, 8, 0, 4, 256, defaultMedianLevel);
 
     }
 

src/ripple/rpc/handlers/SignFor.cpp

@@ -50,8 +50,7 @@ Json::Value doSignFor (RPC::Context& context)
         failType,
         context.role,
         context.ledgerMaster.getValidatedLedgerAge(),
-        context.app,
+        context.app);
-        context.ledgerMaster.getCurrentLedger());
 }
 
 } // ripple

src/ripple/rpc/handlers/SignHandler.cpp

@@ -43,8 +43,7 @@ Json::Value doSign (RPC::Context& context)
         failType,
         context.role,
         context.ledgerMaster.getValidatedLedgerAge(),
-        context.app,
+        context.app);
-        context.ledgerMaster.getCurrentLedger());
 }
 
 } // ripple

src/ripple/rpc/handlers/Submit.cpp

@@ -55,7 +55,6 @@ Json::Value doSubmit (RPC::Context& context)
         context.role,
         context.ledgerMaster.getValidatedLedgerAge(),
         context.app,
-        context.ledgerMaster.getCurrentLedger(),
         RPC::getProcessTxnFn (context.netOps));
     }
 

src/ripple/rpc/handlers/SubmitMultiSigned.cpp

@@ -50,7 +50,6 @@ Json::Value doSubmitMultiSigned (RPC::Context& context)
         context.role,
         context.ledgerMaster.getValidatedLedgerAge(),
         context.app,
-        context.ledgerMaster.getCurrentLedger(),
         RPC::getProcessTxnFn (context.netOps));
 }
 

src/ripple/rpc/impl/TransactionSign.cpp

@@ -20,8 +20,10 @@
 #include <BeastConfig.h>
 #include <ripple/rpc/impl/TransactionSign.h>
 #include <ripple/app/ledger/LedgerMaster.h>
+#include <ripple/app/ledger/OpenLedger.h>
 #include <ripple/app/main/Application.h>
 #include <ripple/app/misc/Transaction.h>
+#include <ripple/app/misc/TxQ.h>
 #include <ripple/app/paths/Pathfinder.h>
 #include <ripple/app/tx/apply.h>              // Validity::Valid
 #include <ripple/basics/Log.h>
@@ -31,6 +33,7 @@
 #include <ripple/net/RPCErr.h>
 #include <ripple/protocol/Sign.h>
 #include <ripple/protocol/ErrorCodes.h>
+#include <ripple/protocol/Feature.h>
 #include <ripple/protocol/STAccount.h>
 #include <ripple/protocol/STParsedJSON.h>
 #include <ripple/protocol/TxFlags.h>
@@ -339,7 +342,7 @@ transactionPreProcessImpl (
     SigningForParams& signingArgs,
     std::chrono::seconds validatedLedgerAge,
     Application& app,
-    std::shared_ptr<ReadView const> const& ledger)
+    std::shared_ptr<OpenView const> const& ledger)
 {
     auto j = app.journal ("RPCHandler");
 
@@ -387,12 +390,13 @@ transactionPreProcessImpl (
     }
 
     {
-        Json::Value err = checkFee (
+        Json::Value err = checkFee(
             params,
             role,
             verify && signingArgs.editFields(),
             app.config(),
             app.getFeeTrack(),
+            app.getTxQ(),
             ledger);
 
         if (RPC::contains_error (err))
@@ -621,13 +625,14 @@ static Json::Value transactionFormatResultImpl (Transaction::pointer tpTrans)
 
 //------------------------------------------------------------------------------
 
-Json::Value checkFee (
+Json::Value checkFee(
     Json::Value& request,
     Role const role,
     bool doAutoFill,
     Config const& config,
     LoadFeeTrack const& feeTrack,
-    std::shared_ptr<ReadView const> const& ledger)
+    TxQ const& txQ,
+    std::shared_ptr<OpenView const> const& ledger)
 {
     Json::Value& tx (request[jss::tx_json]);
     if (tx.isMember (jss::Fee))
@@ -670,14 +675,38 @@ Json::Value checkFee (
     std::uint64_t const feeDefault = config.TRANSACTION_FEE_BASE;
 
     // Administrative and identified endpoints are exempt from local fees.
-    std::uint64_t const fee =
+    std::uint64_t const loadFee =
         feeTrack.scaleFeeLoad (feeDefault,
             ledger->fees().base, ledger->fees().units, isUnlimited (role));
+    std::uint64_t fee = loadFee;
+    if (ledger->rules().enabled(featureFeeEscalation,
+        config.features))
+    {
+        auto const assumeTx = request.isMember("x-assume-tx") &&
+            request["x-assume-tx"].isConvertibleTo (Json::uintValue) ?
+                request["x-assume-tx"].asUInt() : 0;
+        auto const metrics = txQ.getMetrics(*ledger, assumeTx);
+        auto const baseFee = ledger->fees().base;
+        auto escalatedFee = mulDiv(
+            metrics.expFeeLevel, baseFee,
+                metrics.referenceFeeLevel).second;
+        if (mulDiv(escalatedFee, metrics.referenceFeeLevel,
+                baseFee).second < metrics.expFeeLevel)
+            ++escalatedFee;
+        fee = std::max(fee, escalatedFee);
+    }
 
     auto const limit = mulDivThrow(feeTrack.scaleFeeBase (
         feeDefault, ledger->fees().base, ledger->fees().units),
         mult, div);
 
+    if (fee > limit && fee != loadFee &&
+        request.isMember("x-queue-okay") &&
+            request["x-queue-okay"].isBool() &&
+                request["x-queue-okay"].asBool())
+    {
+        fee = loadFee;
+    }
     if (fee > limit)
     {
         std::stringstream ss;
@@ -698,11 +727,11 @@ Json::Value transactionSign (
     NetworkOPs::FailHard failType,
     Role role,
     std::chrono::seconds validatedLedgerAge,
-    Application& app,
+    Application& app)
-    std::shared_ptr<ReadView const> const& ledger)
 {
     using namespace detail;
 
+    auto const& ledger = app.openLedger().current();
     auto j = app.journal ("RPCHandler");
     JLOG (j.debug) << "transactionSign: " << jvRequest;
 
@@ -733,11 +762,11 @@ Json::Value transactionSubmit (
     Role role,
     std::chrono::seconds validatedLedgerAge,
     Application& app,
-    std::shared_ptr<ReadView const> const& ledger,
     ProcessTransactionFn const& processTransaction)
 {
     using namespace detail;
 
+    auto const& ledger = app.openLedger().current();
     auto j = app.journal ("RPCHandler");
     JLOG (j.debug) << "transactionSubmit: " << jvRequest;
 
@@ -860,9 +889,9 @@ Json::Value transactionSignFor (
     NetworkOPs::FailHard failType,
     Role role,
     std::chrono::seconds validatedLedgerAge,
-    Application& app,
+    Application& app)
-    std::shared_ptr<ReadView const> const& ledger)
 {
+    auto const& ledger = app.openLedger().current();
     auto j = app.journal ("RPCHandler");
     JLOG (j.debug) << "transactionSignFor: " << jvRequest;
 
@@ -975,9 +1004,9 @@ Json::Value transactionSubmitMultiSigned (
     Role role,
     std::chrono::seconds validatedLedgerAge,
     Application& app,
-    std::shared_ptr<ReadView const> const& ledger,
     ProcessTransactionFn const& processTransaction)
 {
+    auto const& ledger = app.openLedger().current();
     auto j = app.journal ("RPCHandler");
     JLOG (j.debug)
         << "transactionSubmitMultiSigned: " << jvRequest;
@@ -1018,7 +1047,8 @@ Json::Value transactionSubmitMultiSigned (
 
     {
         Json::Value err = checkFee (
-            jvRequest, role, false, app.config(), app.getFeeTrack(), ledger);
+            jvRequest, role, false, app.config(), app.getFeeTrack(),
+                app.getTxQ(), ledger);
 
         if (RPC::contains_error(err))
             return err;

src/ripple/rpc/impl/TransactionSign.h

@@ -30,6 +30,7 @@ namespace ripple {
 class Application;
 class LoadFeeTrack;
 class Transaction;
+class TxQ;
 
 namespace RPC {
 
@@ -66,7 +67,8 @@ Json::Value checkFee (
     bool doAutoFill,
     Config const& config,
     LoadFeeTrack const& feeTrack,
-    std::shared_ptr<ReadView const> const& ledger);
+    TxQ const& txQ,
+    std::shared_ptr<OpenView const> const& ledger);
 
 // Return a std::function<> that calls NetworkOPs::processTransaction.
 using ProcessTransactionFn =
@@ -88,8 +90,7 @@ Json::Value transactionSign (
     NetworkOPs::FailHard failType,
     Role role,
     std::chrono::seconds validatedLedgerAge,
-    Application& app,
+    Application& app);
-    std::shared_ptr<ReadView const> const& ledger);
 
 /** Returns a Json::objectValue. */
 Json::Value transactionSubmit (
@@ -98,7 +99,6 @@ Json::Value transactionSubmit (
     Role role,
     std::chrono::seconds validatedLedgerAge,
     Application& app,
-    std::shared_ptr<ReadView const> const& ledger,
     ProcessTransactionFn const& processTransaction);
 
 /** Returns a Json::objectValue. */
@@ -107,8 +107,7 @@ Json::Value transactionSignFor (
     NetworkOPs::FailHard failType,
     Role role,
     std::chrono::seconds validatedLedgerAge,
-    Application& app,
+    Application& app);
-    std::shared_ptr<ReadView const> const& ledger);
 
 /** Returns a Json::objectValue. */
 Json::Value transactionSubmitMultiSigned (
@@ -117,7 +116,6 @@ Json::Value transactionSubmitMultiSigned (
     Role role,
     std::chrono::seconds validatedLedgerAge,
     Application& app,
-    std::shared_ptr<ReadView const> const& ledger,
     ProcessTransactionFn const& processTransaction);
 
 } // RPC

src/ripple/rpc/tests/JSONRPC.test.cpp

@@ -18,6 +18,7 @@
 //==============================================================================
 
 #include <BeastConfig.h>
+#include <ripple/app/misc/TxQ.h>
 #include <ripple/basics/contract.h>
 #include <ripple/core/LoadFeeTrack.h>
 #include <ripple/json/json_reader.h>
@@ -1748,10 +1749,8 @@ public:
     void testAutoFillFees ()
     {
         test::jtx::Env env(*this);
-        std::shared_ptr<const ReadView> ledger =
+        auto ledger = env.current();
-            std::make_shared<Ledger>(create_genesis,
+        LoadFeeTrack const& feeTrack = env.app().getFeeTrack();
-                env.app().config(), env.app().family());
-        LoadFeeTrack const feeTrack;
 
         {
             Json::Value req;
@@ -1759,9 +1758,12 @@ public:
                 "{ \"fee_mult_max\" : 1, \"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee (req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect (! RPC::contains_error (result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
         }
 
         {
@@ -1771,9 +1773,12 @@ public:
                     "\"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee(req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
         }
 
         {
@@ -1782,9 +1787,11 @@ public:
                 "{ \"fee_mult_max\" : 0, \"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee (req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect (RPC::contains_error (result), "Invalid checkFee");
+            expect(!req[jss::tx_json].isMember(jss::Fee));
         }
 
         {
@@ -1796,9 +1803,11 @@ public:
                     "\"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee(req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect(RPC::contains_error(result), "Invalid checkFee");
+            expect(!req[jss::tx_json].isMember(jss::Fee));
         }
 
         {
@@ -1808,9 +1817,11 @@ public:
                     "\"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee(req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect(RPC::contains_error(result), "Invalid checkFee");
+            expect(!req[jss::tx_json].isMember(jss::Fee));
         }
 
         {
@@ -1820,12 +1831,143 @@ public:
                     "\"tx_json\" : { } } ", req);
             Json::Value result =
                 checkFee(req, Role::ADMIN, true,
-                    env.app().config(), feeTrack, ledger);
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), ledger);
 
             expect(RPC::contains_error(result), "Divide by 0");
+            expect(!req[jss::tx_json].isMember(jss::Fee));
         }
     }
 
+    void testAutoFillEscalatedFees ()
+    {
+        test::jtx::Env env(*this,
+            test::jtx::features(featureFeeEscalation));
+        LoadFeeTrack const& feeTrack = env.app().getFeeTrack();
+        env.app().getTxQ().setMinimumTx(3);
+
+        {
+            // 1: high mult, no queue, no pad
+            Json::Value req;
+            Json::Reader ().parse (
+                "{ \"fee_mult_max\" : 1000, \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee (req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect (! RPC::contains_error (result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
+        }
+
+        {
+            // 2: high mult, can queue, no pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 1000, \"x-queue-okay\" : true \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
+        }
+
+        {
+            // 3: high mult, no queue, 4 pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 1000, \"x-assume-tx\" : 4, \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 8889);
+        }
+
+        {
+            // 4: high mult, can queue, 4 pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 1000, \"x-assume-tx\" : 4, \"x-queue-okay\" : true \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 8889);
+        }
+
+        ///////////////////
+        {
+            // 5: low mult, no queue, no pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 5, \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
+        }
+
+        {
+            // 6: low mult, can queue, no pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 5, \"x-queue-okay\" : true \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                      env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
+        }
+
+        {
+            // 7: low mult, no queue, 4 pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 5, \"x-assume-tx\" : 4, \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                      env.app().getTxQ(), env.current());
+
+            expect(RPC::contains_error(result), "Invalid checkFee");
+            expect(!req[jss::tx_json].isMember(jss::Fee));
+        }
+
+        {
+            // 8: : low mult, can queue, 4 pad
+            Json::Value req;
+            Json::Reader().parse(
+                "{ \"fee_mult_max\" : 5, \"x-assume-tx\" : 4, \"x-queue-okay\" : true \"tx_json\" : { } } ", req);
+            Json::Value result =
+                checkFee(req, Role::ADMIN, true,
+                    env.app().config(), feeTrack,
+                        env.app().getTxQ(), env.current());
+
+            expect(!RPC::contains_error(result), "Legal checkFee");
+            expect(req[jss::tx_json].isMember(jss::Fee) &&
+                req[jss::tx_json][jss::Fee] == 10);
+        }
+
+    }
+
     // A function that can be called as though it would process a transaction.
     static void fakeProcessTransaction (
         std::shared_ptr<Transaction>&, bool, bool, NetworkOPs::FailHard)
@@ -1854,8 +1996,6 @@ public:
         env(pay(g, env.master, USD(50)));
         env.close();
 
-        auto const ledger = env.current();
-
         ProcessTransactionFn processTxn = fakeProcessTransaction;
 
         // A list of all the functions we want to test.
@@ -1864,8 +2004,7 @@ public:
             NetworkOPs::FailHard failType,
             Role role,
             std::chrono::seconds validatedLedgerAge,
-            Application& app,
+            Application& app);
-            std::shared_ptr<ReadView const> const& ledger);
 
         using submitFunc = Json::Value (*) (
             Json::Value params,
@@ -1873,7 +2012,6 @@ public:
             Role role,
             std::chrono::seconds validatedLedgerAge,
             Application& app,
-            std::shared_ptr<ReadView const> const& ledger,
             ProcessTransactionFn const& processTransaction);
 
         using TestStuff =
@@ -1913,8 +2051,7 @@ public:
                             NetworkOPs::FailHard::yes,
                             testRole,
                             1s,
-                            env.app(),
+                            env.app());
-                            ledger);
                     }
                     else
                     {
@@ -1926,7 +2063,6 @@ public:
                             testRole,
                             1s,
                             env.app(),
-                            ledger,
                             processTxn);
                     }
 
@@ -1947,6 +2083,7 @@ public:
     void run ()
     {
         testAutoFillFees ();
+        testAutoFillEscalatedFees ();
         testTransactionRPC ();
     }
 };