diff --git a/README.md b/README.md index df49d198d9..ccdfb71cc2 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,28 @@ ripple-dev-portal ================= -The [Ripple Developer Portal](https://dev.ripple.com) is the authoritative source for Ripple documentation, including the `rippled` server, Ripple-REST API, Gatewayd, and other Ripple software. +The [Ripple Developer Portal](https://dev.ripple.com) is the authoritative source for Ripple documentation, including the `rippled` server, RippleAPI, the Ripple Data API, and other Ripple software. Repository Layout ----------------- -The pages in this portal are generated from the markdown files in the [content/](content/) folder. +The HTML pages in this portal are generated from the markdown files in the [content/](content/) folder. -You can run your own local copy of the Dev Portal by hosting it in a web server, such as Apache. By default, the Dev Portal uses static HTML pages and client-side JavaScript (specifically, [Flatdoc](http://ricostacruz.com/flatdoc/)) to parse and display the Markdown content on the fly. +The [tool/](tool/) folder contains tools and templates for generating the HTML files in the top level. The `parse_pages.py` script (requires Python 3, [Jinja 2](http://jinja.pocoo.org/), and various pip modules) uses the templates and the [pages.json](tools/pages.json) file to generate all the HTML files on the top level. Files with "md" properties are considered documentation pages and appear in the appropriate lists. In general, `parse_pages.py` assumes you are running it with `tool/` as the current working directory. -The [tool/](tool/) folder contains tools and templates for generating the HTML files in the top level. The `parse_pages.py` script (Python 3, [Jinja 2](http://jinja.pocoo.org/), and various pip modules required) uses the templates and the [pages.json](tools/pages.json) file to generate all the HTML files on the top level. Files with "md" properties are considered documentation pages and appear in the appropriate lists. There are a few modes for generating pages: +There are a few modes for generating pages: -* The default, dynamic pages. Generates thin HTML wrappers for each page and uses Flatdoc to load and parse the contents. Great for development since a refresh automatically picks up changes to the md files. -* Pre-compiled pages. Parses the Markdown all at once and puts the contents directly into the HTML body of each file. Still experimental. -* PDF output using [Prince](http://www.princexml.com/). Still experimental. +* By default, parses the Markdown and generates all the HTML files for a locally-hosted environment. + * Local markdown files are pre-processed using Jinja so you they can have conditional logic + * You can also remotely reference a markdown file via URL. Such files are not pre-processed, though. +* Build for a specific target (e.g. `./parse_pages.py -t ripple.com`). Replaces local links with alternate versions for the target, uncomments multi-code divs (if target needs them for multi-code tabs), and performs various other cleanup. +* Watch for changes and re-build the HTML files automatically when changes occur. Example: `./parse_pages.py -w` +* Output empty HTML files and use [Flatdoc](ricostacruz.com/flatdoc/) to parse the markdown in-browser. Requires that you host the dev portal in a web server (e.g. Apache), not just open the HTML files in a browser. Example: `./parse_pages.py --flatdoc` +* Output a modified markdown file for GitHub (e.g. for sending changes back upstream). (Builds for a target, but stops at markdown instead of HTML.) e.g. `./parse_pages.py -t ripple.com -g data_v2.md > /tmp/data_v2_githubified.md` +* PDF output using [Prince](http://www.princexml.com/). Use `./parse_pages.py --pdf some_out_file.pdf`. Experimental. -To add more pages to the Dev Portal, simply modify `pages.json` and then run `parse_pages.py` again. +To add more pages to the Dev Portal, simply modify `pages.json` and then run `parse_pages.py` again. For full usage, run `parse_pages.py --help`. Contributing @@ -25,5 +30,5 @@ Contributing The Developer Portal welcomes outside contributions, especially to the documentation contents. If you have any corrections, improvements, or expansions of the portal, please contribute pull requests to the **gh-pages** branch. -Contributions become copyright Ripple Labs and are provided under the MIT [LICENSE](LICENSE). +Contributions become copyright Ripple, Inc. and are provided under the MIT [LICENSE](LICENSE). diff --git a/tool/githubify.py b/tool/githubify.py deleted file mode 100755 index 1f143cc7c0..0000000000 --- a/tool/githubify.py +++ /dev/null @@ -1,49 +0,0 @@ -#!/bin/env python3 -""" -Convert between Dev-Portal-ready markdown and Github-ready markdown. Has two modes: -Normal - Convert from Dev Portal format to Github: - * Comments out divs that the dev portal uses to generate tabs, so that Github parses the markdown inside - * Replaces local relative links with absolute links to ripple.com -Reverse - Convert from Github format to Dev Portal: - * Uncomments multicode divs - * Replaces absolute links with local ones that will work even offline. - -Usage: githubify.py ripplerest_api.md > readme.md - -You may still want to do some manual editing (for example, to add Travis status icons to your readme) -""" - -import sys - -def convert_page(text, reverse): - replacements = { - "
": "", - "
": "", - "(rest-api-tool.html": "(https://ripple.com/build/rest-tool", - "(transactions.html": "(https://ripple.com/build/transactions", - "(rippled-apis.html": "(https://ripple.com/build/rippled-apis", - "(charts-api-tool.html": "(https://ripple.com/build/charts-api-tool/", - } - - for (k,v) in replacements.items(): - if reverse: - text = text.replace(v,k) - else: - text = text.replace(k,v) - - return text - -if __name__ == "__main__": - if len(sys.argv) < 2 or len(sys.argv) > 3: - exit("usage: %s infile [reverse]" % sys.argv[0]) - - if len(sys.argv) == 3 and sys.argv[2].lower() == "reverse": - reverse = True - else: - reverse = False - - with open(sys.argv[1]) as f: - text = f.read() - text = convert_page(text, reverse) - print(text) - diff --git a/tool/parse_pages.py b/tool/parse_pages.py index bd7d444c50..acf5e938c4 100755 --- a/tool/parse_pages.py +++ b/tool/parse_pages.py @@ -121,9 +121,52 @@ def parse_markdown(md, target=DEFAULT_TARGET, pages=None): html2 = str(soup) print("done") return html2 + +MARKDOWN_LINK_REGEX = re.compile(r"(\[([^\]]+)\]\(([^:)]+)\)|\[([^\]]+)\]:\s*(\S+)$)", re.MULTILINE) +def githubify_markdown(md, target=DEFAULT_TARGET, pages=None): + if not pages: + pages = get_pages() + + class MDLink: + def __init__(self, fullmatch, label, url, label2, url2): + self.fullmatch = fullmatch + if label: + self.label = label + self.url = url + self.is_reflink = False + elif label2: + self.label = label2 + self.url = url2 + self.is_reflink = True + + def to_markdown(self): + s = "[" + s += self.label + s += "]" + if self.is_reflink: + s += ": " + s += self.url + else: + s += "(" + s += self.url + s += ")" + return s + + links = [MDLink(*m) for m in MARKDOWN_LINK_REGEX.findall(md)] -def get_pages(target=None): - print("reading page manifest...") + for link in links: + for page in pages: + if target in page: + #There's a replacement link for this + local_url = page["html"] + target_url = page[target] + if link.url[:len(local_url)] == local_url: + link.url = link.url.replace(local_url, target_url) + md = md.replace(link.fullmatch, link.to_markdown()) + + return md + +def get_pages(target=None,verbose=True): with open(PAGE_MANIFEST_FILE) as f: pages = json.load(f) @@ -132,7 +175,6 @@ def get_pages(target=None): pages = [page for page in pages if "targets" not in page or target in page["targets"] ] - print("done") return pages def render_pages(precompiled, target=DEFAULT_TARGET): @@ -271,6 +313,15 @@ def make_pdf(outfile): print("generating PDF: running ", " ".join(args),"...") prince_resp = subprocess.check_output(args, universal_newlines=True) + +def githubify(md_file_name, target=DEFAULT_TARGET): + filein = os.path.join(CONTENT_PATH, md_file_name) + with open(filein, "r") as f: + md = f.read() + pages = get_pages() + print(githubify_markdown(md, target=target, pages=pages)) + + if __name__ == "__main__": parser = argparse.ArgumentParser( description='Generate static site from markdown and templates.') @@ -280,10 +331,15 @@ if __name__ == "__main__": help="Watch for changes and re-generate the files. This runs until force-quit.") parser.add_argument("--pdf", type=str, help="Generate a PDF, too. Requires Prince.") + parser.add_argument("-g","--githubify", type=str, help="Output md prepared for GitHub") parser.add_argument("--target", "-t", type=str, default=DEFAULT_TARGET) args = parser.parse_args() pre_parse = not args.flatdoc + if args.githubify: + githubify(args.githubify, args.target) + exit(0) + if args.pdf: if args.pdf[-4:] != ".pdf": exit("PDF filename must end in .pdf")