12890 Commits

Author SHA1 Message Date
Ed Hennis
9b2d563dec fix: support RPC markers for any ledger object: (#4361)
There were situations where `marker`s  returned by `account_lines` did
not work on subsequent requests, returning "Invalid Parameters".

This was caused by the optimization implemented in "Enforce account RPC
limits by account objects traversed":

e28989638d

Previously, the ledger traversal would find up to `limit` account lines,
and if there were more, the marker would be derived from the key of the
next account line. After the change, ledger traversal would _consider_
up to `limit` account objects of any kind found in the account's
directory structure. If there were more, the marker would be derived
from the key of the next object, regardless of type.

With this optimization, it is expected that `account_lines` may return
fewer than `limit` account lines - even 0 - along with a marker
indicating that there are may be more available.

The problem is that this optimization did not update the
`RPC::isOwnedByAccount` helper function to handle those other object
types. Additionally, XLS-20 added `ltNFTOKEN_OFFER` ledger objects to
objects that have been added to the account's directory structure, but
did not update `RPC::isOwnedByAccount` to be able to handle those
objects. The `marker` provided in the example for #4354 includes the key
for an `ltNFTOKEN_OFFER`. When that `marker` is used on subsequent
calls, it is not recognized as valid, and so the request fails.

* Add unit test that walks all the object types and verifies that all of
  their indexes can work as a marker.
* Fix #4340
* Fix #4354
2023-03-20 10:22:15 -07:00
Nik Bougalis
150d4a47e4 refactor: optimize NodeStore object conversion: (#4353)
When writing objects to the NodeStore, we need to convert them from
the in-memory format to the binary format used by the node store.

The conversion is handled by the `EncodedBlob` class, which is only
instantiated on the stack. Coupled with the fact that most objects
are under 1024 bytes in size, this presents an opportunity to elide
a memory allocation in a critical path.

This commit also simplifies the interface of `EncodedBlob` and
eliminates a subtle corner case that could result in dangling
pointers.

These changes are not expected to cause a significant reduction in
memory usage. The change avoids the use of a `std::shared_ptr` when
unnecessary and tries to use stack-based memory allocation instead
of the heap whenever possible.

This is a net gain both in terms of memory usage (lower
fragmentation) and performance (less work to do at runtime).
2023-03-16 15:00:07 -07:00
Ed Hennis
1c9df69b33 fix(ValidatorSite): handle rare null pointer dereference in timeout: (#4420)
In rare circumstances, both `onRequestTimeout` and the response handler
(`onSiteFetch` or `onTextFetch`) can get queued and processed. In all
observed cases, the response handler processes a network error.
`onRequestTimeout` usually runs first, but on rare occasions, the
response handler runs first, which leaves `activeResource` empty.
2023-03-16 10:32:22 -07:00
RichardAH
10555faa92 fix(gateway_balances): handle overflow exception: (#4355)
* Prevent internal error by catching overflow exception in `gateway_balances`.
* Treat `gateway_balances` obligations overflow as max (largest valid) `STAmount`.
  * Note that very large sums of STAmount are approximations regardless.

---------

Co-authored-by: Scott Schurr <scott@ripple.com>
2023-03-16 10:25:40 -07:00
Elliot Lee
0f1ffff068 Set version to 1.10.1-b1 2023-03-14 21:21:50 -07:00
Chenna Keshava B S
9309b57364 Rectify the import paths of boost/iterator: (#4293)
- MSVC 19.x reported a warning about import paths in boost for
  function_output_iterator class (boost::function_output_iterator).
- Eliminate that warning by updating the import paths, as suggested by
  the compiler warnings.
2023-03-14 21:10:56 -07:00
RichardAH
cb08f2b6ec Allow port numbers be be specified with a colon: (#4328)
Port numbers can now be specified using either a colon or a space.

Examples:

1.2.3.4:51235

1.2.3.4 51235

- In the configuration file, an annoying "gotcha" for node operators is
  accidentally specifying IP:PORT combinations using a colon. The code
  previously expected a space, not a colon. It also does not provide
  good feedback when this operator error is made.
- This change simply allows this mistake (using a colon) to be fixed
  automatically, preserving the intention of the operator.
- Add unit tests, which test the functionality when specifying IP:PORT
  in the configuration file.
- The RPCCall test regime is not specific enough to test this
  functionality, it has been tested by hand.
- Ensure IPv6 addresses are not confused for ip:port

---------

Co-authored-by: Elliot Lee <github.public@intelliot.com>
2023-03-14 21:06:30 -07:00
drlongle
84cde3ce0b Use <=> operator for base_uint, Issue, and Book: (#4411)
- Implement the `operator==` and the `operator<=>` (aka the spaceship
  operator) in `base_uint`, `Issue`, and `Book`. 
- C++20-compliant compilers automatically provide the remaining
  comparison operators (e.g. `operator<`, `operator<=`, ...).
- Remove the function compare() because it is no longer needed.
- Maintain the same semantics as the existing code.
- Add some unit tests to gain further confidence.
- Fix #2525.
2023-03-14 20:54:54 -07:00
Mark Travis
f7b3ddd87b Reporting Mode: Do not attempt to acquire missing data from peer network (#4458)
In Reporting Mode, a server would core dump when it is not able to read
from Cassandra. This patch prevents the core dump when Cassandra is down
for reporting mode servers. This does not fix the root cause, but it
cuts down on some of the resulting noise.
2023-03-14 20:49:40 -07:00
Elliot Lee
1e7710eee2 docs: security bug bounty acknowledgements (#4460) 2023-03-14 13:08:56 -07:00
Elliot Lee
07f047b1e2 Set version to 1.10.0
Merge #4451
1.10.0
2023-03-14 09:30:22 -07:00
Elliot Lee
8687b5c3c9 Set version to 1.10.0-rc4 1.10.0-rc4 2023-03-02 14:28:08 -08:00
Kenny Lei
ecd49e1535 Rename 'NFT' to 'NFToken' in DisallowIncoming flags (#4442)
* Follow-up to #4336
* NFToken is the naming convention in the codebase (rather than NFT)
* Rename `lsfDisallowIncomingNFTOffer` to `lsfDisallowIncomingNFTokenOffer`
* Rename `asfDisallowIncomingNFTOffer` to `asfDisallowIncomingNFTokenOffer`
2023-03-02 13:01:50 -08:00
Elliot Lee
c77a8d5ec6 Update Docker.md (#4432)
* Add links to some related resources that may be helpful.
* Docker images can make testing easier to do.
2023-03-02 10:07:09 -08:00
Michael Legleux
e13676f709 Update package building scripts and images to use Conan (#4435)
* Set version to 1.10.0-rc3
* Test on Fedora 37
* Dependency builds are handled by Conan
2023-02-28 14:17:07 -08:00
Elliot Lee
74594d5348 Disable duplicate detector: (#4438)
Partially revert the functionality introduced
with #4195 / 5a15229 (part of 1.10.0-b1).

Acknowledgements:
Aaron Hook for responsibly disclosing this issue.

Bug Bounties and Responsible Disclosures:
We welcome reviews of the rippled code and urge researchers to
responsibly disclose any issues they may find.

To report a bug, please send a detailed report to:

    bugs@xrpl.org

---------

Co-authored-by: Nik Bougalis <nikb@bougalis.net>
2023-02-28 14:12:28 -08:00
Ed Hennis
caf4827c0b Undo API changes introduced in XRPFees: (#4429)
* Original changes: e4b17d1cf2
* Resolve #4425
2023-02-24 08:41:28 -08:00
Elliot Lee
c2b03fecca Remove recipe for RocksDB and add recipe for Snappy (#4431)
- Copies the recipe for Snappy from Conan Center, but removes three
  lines that explicitly link the standard library, which prevents
  builders from statically linking it.
- Removes the recipe for RocksDB now that an official recipe for version
  6.27.3 is in Conan Center.

Developers will likely need to remove cached versions of both RocksDB
and Snappy:

```
conan remove -f rocksdb
conan remove -f snappy
```

---------

Co-authored-by: John Freeman <jfreeman08@gmail.com>
2023-02-23 12:41:18 -08:00
John Freeman
60c276d90b Fix Conan version constraint in workflows (#4430)
Conan 2.0 is not compatible with the current workflows.
2023-02-22 19:54:51 -08:00
Levin Winter
2929748898 Refactor getTrustedForLedger() (#4424)
Look for validations associated with a specific ledger ID and sequence
number.
2023-02-22 15:20:15 -08:00
Elliot Lee
5ec8783d35 README: Update "Build from source" section (#4426)
Fix #2987
2023-02-21 15:40:09 -08:00
Elliot Lee
96aab1288f README: Add a few source code starting points (#4421)
Co-authored-by: John Freeman <jfreeman08@gmail.com>
2023-02-17 22:27:27 -08:00
Ed Hennis
aebf2ac990 Resolve a couple of Github Action CI annoyances: (#4413)
* Set "fail-fast: false" so that multiple jobs in one workflow can
  finish independently. By default, if one job fails, other running jobs
  will be aborted, even if the other jobs are working fine and are
  almost done. This leads to wasted time and resources if the failure
  is, for example, OS specific, or due to a flaky unit test, and the
  failed job needs to be re-run, because all the jobs end up re-running.
* Put conditions back into the windows.yml job (manual, and for
  a specific branch name and that job). This prevents Github Actions
  from sending "No jobs were run" failure emails on every commit.
2023-02-15 11:34:32 -08:00
ledhed2222
ac78b7a9a7 Rename to fixNonFungibleTokensV1_2 and some cosmetic changes (#4419) 2023-02-13 15:52:40 -08:00
Denis Angell
b72a87c7d3 Only account specified as destination can settle through brokerage: (#4399)
Without this amendment, for NFTs using broker mode, if the sell offer contains a destination and that destination is the buyer account, anyone can broker the transaction. Also, if a buy offer contains a destination and that destination is the seller account, anyone can broker the transaction. This is not ideal and is misleading.

Instead, with this amendment: If you set a destination, that destination needs to be the account settling the transaction. So, the broker must be the destination if they want to settle. If the buyer is the destination, then the buyer must accept the sell offer, as you cannot broker your own offers.

If users want their offers open to the public, then they should not set a destination. On the other hand, if users want to limit who can settle the offers, then they would set a destination.

Unit tests:

1. The broker cannot broker a destination offer to the buyer and the buyer must accept the sell offer. (0 transfer)
2. If the broker is the destination, the broker will take the difference. (broker mode)
2023-02-13 15:52:39 -08:00
Scott Schurr
39c32561bd Prevent brokered sale of NFToken to owner: (#4403)
Fixes #4374

It was possible for a broker to combine a sell and a buy offer from an account that already owns an NFT. Such brokering extracts money from the NFT owner and provides no benefit in return.

With this amendment, the code detects when a broker is returning an NFToken to its initial owner and prohibits the transaction. This forbids a broker from selling an NFToken to the account that already owns the token. This fixes a bug in the original implementation of XLS-20.

Thanks to @nixer89 for suggesting this fix.
2023-02-13 15:52:39 -08:00
ledhed2222
89aa8b21ec Fix 3 issues around NFToken offer acceptance (#4380)
Fixes 3 issues:

In the following scenario, an account cannot perform NFTokenAcceptOffer even though it should be allowed to:

- BROKER has < S
- ALICE offers to sell token for S
- BOB offers to buy token for > S
- BROKER tries to bridge the two offers

This currently results in `tecINSUFFICIENT_FUNDS`, but should not because BROKER is not spending any funds in this transaction, beyond the transaction fee.

When trading an NFT using IOUs, and when the issuer of the IOU has any non-zero value set for TransferFee on their account via AccountSet (not a TransferFee on the NFT), and when the sale amount is equal to the total balance of that IOU that the buyer has, the resulting balance for the issuer of the IOU will become positive. This means that the buyer of the NFT was supposed to have caused a certain amount of IOU to be burned. That amount was unable to be burned because the buyer couldn't cover it. This results in the buyer owing this amount back to the issuer. In a real world scenario, this is appropriate and can be settled off-chain.

Currency issuers could not make offers for NFTs using their own currency, receiving `tecINSUFFICIENT_FUNDS` if they tried to do so.

With this fix, they are now able to buy/sell NFTs using their own currency.
2023-02-13 15:52:39 -08:00
Shawn Xie
a828e24cf0 Allow NFT to be burned when number of offers is greater than 500 (#4346)
* Allow offers to be removable
* Delete sell offers first

Signed-off-by: Shawn Xie <shawnxie920@gmail.com>
2023-02-13 15:52:38 -08:00
ledhed2222
f7a8d2de84 Add fixUnburnableNFToken feature (#4391) 2023-02-13 15:52:38 -08:00
Howard Hinnant
32559463ef Change default vote on fixUniversalNumber from yes to no (#4414) 2023-02-09 21:01:48 -08:00
Ed Hennis
8f514937a4 Update documented pathfinding configuration defaults: (#4409)
* Add recommended values for node which want to support "advanced"
  pathfinding, which is effectively the old default behavior.
2023-02-08 12:14:09 -08:00
John Freeman
36b34a7bd5 Update dependency: grpc (#4407) 2023-02-07 15:46:38 -08:00
Howard Hinnant
5edaec2bd0 Introduce min/max observers for Number
Three static member functions are introduced with
definitions consistent with std::numeric_limits:

static constexpr Number min() noexcept;

  Returns: The minimum positive value.  This is the value closest to zero.

static constexpr Number max() noexcept;

  Returns: The maximum possible value.

static constexpr Number lowest() noexcept;

  Returns: The negative value which is less than all other values.
2023-02-07 15:43:28 -08:00
Howard Hinnant
2f1f453052 Optimize uint128_t division by 10 within Number.cpp
* Optimization includes computing remainder from division.
* Used only within Number::operator*=.
2023-02-07 15:43:28 -08:00
Howard Hinnant
6eaaa7bcfa Replace Number division algorithm
* Replace division with faster algorithm.
* Correct some rounding bugs in multiplication.
* Add tests for rounding bugs.
2023-02-07 15:43:28 -08:00
Howard Hinnant
e354497f63 Include rounding mode in XRPAmount to STAmount conversion. 2023-02-07 15:43:28 -08:00
Howard Hinnant
6fcd654bee Remove undefined behavior
* Taking the negative of a signed negative is UB, but
  taking the negative of an unsigned is not.
2023-02-07 15:43:28 -08:00
Howard Hinnant
d275a2ab72 Silence warnings 2023-02-07 15:43:28 -08:00
Howard Hinnant
3f33471220 Introduce rounding modes for Number:
You can set a thread-local flag to direct Number how to round
non-exact results with the syntax:

    Number::rounding_mode prev_mode = Number::setround(Number::towards_zero);

This flag will stay in effect for this thread only until another call
to setround.  The previously set rounding mode is returned.

You can also retrieve the current rounding mode with:

    Number::rounding_mode current_mode = Number::getround();

The available rounding modes are:

* to_nearest : Rounds to nearest representable value.  On tie, rounds
               to even.
* towards_zero : Rounds towards zero.
* downward : Rounds towards negative infinity.
* upward : Rounds towards positive infinity.

The default rounding mode is to_nearest.
2023-02-07 15:43:28 -08:00
Howard Hinnant
a82ad5ba76 Use Number for IOUAmount and STAmount arithmetic
* Guarded by amendment fixUniversalNumber
* Produces slightly better accuracy in some computations.
2023-02-07 15:43:28 -08:00
Howard Hinnant
48e804c40c Add tests 2023-02-07 15:43:28 -08:00
Howard Hinnant
c9c54c9799 Add implicit conversion from STAmount to Number 2023-02-07 15:43:28 -08:00
Howard Hinnant
24fe5f9fd0 Add clip
* Return 0 if abs(x) < limit, else returns x
2023-02-07 15:43:28 -08:00
Howard Hinnant
476ee8a479 Add conversions between Number, XRPAmount and int64_t
* Conversions to Number are implicit
* Conversions away from Number are explicit and potentially lossy
* If lossy, round to nearest, and to even on tie
2023-02-07 15:43:28 -08:00
Howard Hinnant
0ee63b7c7b AMM Add Number class and associated algorithms 2023-02-07 15:43:28 -08:00
John Freeman
31e7e5a56e Revise CONTRIBUTING (#4382)
- Wrap text at 80 columns.
- Match capitalization of GitHub usernames.
- Prescribe more rules for pull requests.
- Link more reference documentation.
2023-02-05 16:08:26 -08:00
Ed Hennis
e4b17d1cf2 XRPFees: Fee setting and handling improvements (#4247)
* Introduces amendment `XRPFees`
* Convert fee voting and protocol messages to use XRPAmounts
* Includes Validations, Change transactions, the "Fees" ledger object,
  and subscription messages

* Improve handling of 0 drop reference fee with TxQ. For use with networks that do not want to require fees
* Note that fee escalation logic is still in place, which may cause the
  open ledger fee to rise if the network is busy. 0 drop transactions
  will still queue, and fee escalation can be effectively disabled by
  modifying the configuration on all nodes

* Change default network reserves to match Mainnet

* Name the new SFields *Drops (not *XRP)
* Reserve SField IDs for Hooks

* Clarify comments explaining the ttFEE transaction field validation
2023-02-02 16:20:35 -08:00
John Freeman
0ce15e0e35 Update BUILD.md (#4383)
Help readers of BUILD.md by moving the crash course to the top
2023-01-11 21:03:38 -08:00
John Freeman
b0e0f319a1 Make NodeToShardRPC a manual test (#4379)
Right now it is flaky, which creates noise in our automated testing.
2023-01-05 18:28:37 -06:00
John Freeman
2233f585f8 Update build instructions (#4376)
Document minimum compiler version; how to choose compiler; how to fix missing `std::result_of`; how to create Conan profile.
2023-01-05 18:27:53 -06:00