Previous query incorrectly pulled and sorted posts, often showing years
old posts from newly followed people. This now behaves more consistently
as "recent threads from people in your extended network".
- Adds a hover popup that shows the names of everyone who liked a post
when hovering of the heart.
- Add new call to post.get that retrieves the names of all voters and
returns them instead of their ID's.
Problem: A `null` relationship represents when it's your profile, which
isn't very intuitive. It causes an error when we try to check for the
`blocking` and `following` properties, which don't exist on `null`.
Solution: Instead of `null`, set a `me` property to tell whether this
relationship is our own profile.
Problem: We have a way to follow and unfollow, but there's no way to
block or unblock. Also if you go to the profile of a blocked peer, their
posts still show up.
Solution: Add block and unblock and hide messages from blocked peers on
their profile page.
Fixes: https://github.com/fraction/oasis/issues/370
Problem: Messages are being fetched with `post.get()`, which runs the
`transform()` function to decorate them with Markdown/etc, but then
they're being passed through `transform()` a second time at the end of
the function. This is inefficient and applies side-effects (like adding
channels to the post) twice.
Solution: Remove the final `transform()` so that these posts are only
decorated once.
Fixes: https://github.com/fraction/oasis/issues/358
Problem: The workaround is no longer necessary now that the new version
of @types/koa has been published. Thanks to @peterblazejewicz,
@harryparkdotio, and @orta for the quick upstream fix.
Solution: Run `npm update` and remove the `@ts-ignore` comment.
Problem: We still have heaps of implicit 'any' types that our linters
can't make sense of.
Solution: Keep on adding type documentation! Slowly but surely I'm
hoping we can reduce the amount of untyped JavaScript in the repo.
Problem: Since we don't validate the referer hostname or the HTTP host
header, an attacker who can trick an Oasis user into clicking a
malicious link can submit HTTP requests to localhost, potentially
publishing messages or extracting information.
Solution: Ensure that all requests have valid hostnames in the referer
to protect against CSRF attacks, ensure that all requests have valid
hostnames in the HTTP header to protect against DNS rebind attacks, and
ensure that errors are only output in the HTTP response when we're sure
that the request is valid.
Problem: After some fantastic new pull requests, there are some
inconsistencies between how each author implemented different features.
Solution: Fix regressions and iron out inconsistencies. This fixes the
heart highlight bug, uses the same indentation for Summaries + Threads +
the thread view, tightens up the CSP now that we don't need
unsafe-inline styles, and uses a neutral tone for indents with the same
width as the blockquote border (so far violet has been reserved for
private messages).
Problem: We're only attempting connection via Unix sockets, but some SSB
services may choose to only listen on TCP sockets.
Solution: Start by attempting connection to Unix socket and fallback to
the TCP socket remote if that doesn't work, before just trying to start
our own SSB service.
While reading long conversations, it's easy to get a bit lost as to whom
is saying what. With some basic CSS, we can keep the header of the post
sticky so you can see more context above the message.
This allow someone to pass in the theme as a command line arg or set the
theme in defaults.json, while still allowing someone to set the theme on
a per-browser basis as a cookie.
Problem: The peer list shows peers that are connected and in the process
of establishing a new connection, which has lots of turbulence. It's
confusing to see dozens of "connections" that only exist for a few
milliseconds when they're actually just connection attempts.
Solution: Only show peers with the state "connected".
Problem: When publishing a root post, other clients give you the option
of adding a `channel` property, which is basically just a hashtag. We
show this at the end of the message, but since replies often copy the
`channel` property then it gets appended to every reply.
Solution: Only append the channel as a hashtag for root posts.
Problem: We have TypeScript enabled on the project but have been missing
lots of definition files, so there are like 500 errors when you enable
`--noImplicitAny`.
Solution: Add definitions and fix the bugs that they point out. This
reduces the number of errors with `--noImplicitAny` to 285.
Problem: When publishing a root post, other clients give you the option
of adding a `channel` property, which is basically just a hashtag. We
show this at the end of the message, but since replies often copy the
`channel` property then it gets appended to every reply.
Solution: Only append the channel as a hashtag for root posts.
Problem: Our dependencies are falling behind and it'd be nice to have
them up-to-date. The only big update it that Common-Good upgraded to the
latest version of Prettier.
Solution: Update them!
Problem: By default we only connect to people that we're directly
following, which often don't have public IP addresses.
Solution: Automatically connect to all available peers within our
replication hops (default 2) to make peering more reliable.
Problem: Trying to view a private message crashes the server and leaks a
bunch of memory. That's bad. This problem is caused by faulty handling
in the function that finds thread ancestors. There's some code that says
"if the next ancestor is a private message, just return the ancestors
that we know about", which returns nothing when we're looking up a
private post (because we can't identify *any* ancestors).
Solution: Ensure that we're only resolving the promise once in the
function by chaining `else if`s and ensure that we only return the
ancestor list if there are actually ancestor in it. If the ancestor list
is empty, we can just return the single message that we know about and
pass it off.
Problem: Since the `errorView()` has a refresh we can probably just call
it `indexingView()` and add the indexing-specific code.
Solution: Rename `errorView()` to `indexingView()` and add a progress
bar for it.
Problem: The indexing message is being thrown as an error, which is
pasted as plaintext, so we can't do a page refresh. This is frustrating
when you're waiting for the indexes to finish because you have to
manually refresh a bunch while you wait.
Solution: Use the prototype from @justinabrahms to add an HTML message
for the indexing error and automatically refresh the page every ten
seconds.
Problem: When our views are still indexing the database they apparently
don't respond over MuxRPC, which means that we're just waiting forever
until they finish. This means that people who are indexing the database
get an HTTP response that might take an hour to finish.
Solution: If we have more than 1 mebibyte of backlog, present an error
message explaining the situation and asking for a bit of patience.
Problem: The SSB-Markdown library has some SSB-flavored quirks, like
messing with newline behavior, and is inappropriate for rendering plain
Markdown like the readme.
Solution: Since SSB-Markdown uses Markdown-It under the hood, we can use
the same library and just render our Markdown *without* the SSB-flavored
quirks. This gives us the ability to wrap text and avoid SSB-Markdown
problems without having to rewrite all of our Markdown.
Problem: A big chunk of both the readme and the startup output is
dedicated to configuration, which feels to me to be focused on
developers and advanced users rather than beginners who just want to try
using Oasis.
Solution: Move readme configuration info to its own file and hide all
config output (except one line) behind `--debug`. While doing this I
noticed that we're `require()`ing a few modules that we don't need
before setting `process.env.DEBUG`, which I've reorganized so that we
don't accidentally disable debug mode for those modules.
Problem: I have debug mode manually enabled and have `console.log()`
calls sprinkled through the code and I'm doing `require("crypto")` too
many times.
Solution: Remove debugging cruft so that we don't have a perma-debug
mode.
Problem: We can set our name and profile description but profile images
are expected by most people and supported by most clients and we don't
have them. A profile without an image can sometimes lack the intimacy
you'd get if you let people upload profile images that they can use as a
visual avatar.
Solution: Add profile image upload to the Edit Profile page and add a
bunch of plumbing for `ssb.blobs.add()` to add the blob and publish a
message setting it as a profile image.
Problem: Other clients render a `channel` property, which is basically a
way to add a single hashtag to your post, but Oasis doesn't support
that. This means that someone might post "this is fun!" under the
channel #running and people using Oasis would just see "this is fun!"
without any context.
Solution: Add the hashtag to the bottom of the post, which visually
looks the same as someone adding two newlines and a hashtag to the end
of the text in their post.
Problem: The Extended page is sorting by received timestamp, not
asserted timestamp.
Solution: Find the `Date.now` and replace it with `Date.now()`, which
refers to the current date instead of `undefined`, which is what you get
if you try to `JSON.stringify(Date.now)`.
Problem: Searching for a hashtag should bring you to the hashtag page,
and the hashtag page should have some useful information about which
page you're on instead of just showing you the messages.
Solution: Add code so that if you search for a hashtag you're brought to
that page, and display some useful help text at the top of the page
explaining the hashtag page. While ensuring that the hashtag page showed
posts from around the network (instead of only people you're following)
I fixed the popular page so that it has the same behavior again (fixing
a regression) and filtered out gatherings (fixing *another* regression)
on the popular page. These probably should've happened in another commit
but I got carried away. :/
Happy to split this commit into two if it hurts to bundle the popular
fixes with the hashtag fix, but I'm low on energy and want to at least
open a PR in case it doesn't bother anyone.
Problem: The search results weren't hiding results from blocked feeds.
Solution: Add `socialFilter()` to ensure that all blocked feeds are
removed from search results as the first step in the stream processing.
Problem: Sometimes you try to run Oasis twice and the second time you
run the program it blows up in your face about the port being taken.
Most other software just focuses or reopens the window instead of
throwing errors at you.
Solution: Implement `/.well-known/oasis` as a sanity check to see
whether the thing listening on the HTTP port. If it replies "oasis" then
we can open the tab (unless `config.open === false`) and avoid throwing
the error.
Problem: An SSB client should allow you to declare your own name, but
Oasis didn't support that behavior at all.
Solution: Add a basic 'Edit Profile' page that lets you set your name
and description. This doesn't implement profile images because I had
limited time and didn't want to think about encoding formats, but it's
worth mentioning that I remember something about binary uploads with the
default form encoding actually send 3 times as much data because of
escapes or something? This might not effect us because we're on
localhost, but I want to make a note that this isn't implemented yet.
This also makes a small change regarding the `<label>` element --
previously we were writing them as siblings to the input and using the
`for` attribute to target the input, but while messing with the CSS I
tried putting them directly in the `label()` and it ends up having the
same effect with less code.
Problem: Trying to use the DTA index when using SSB-Query doesn't work,
because that index only exists on SSB-Backlinks.
Solution: Change the filter to ensure that we're sorted by the asserted
timestamp and ignoring messages from the future. The SSB-Backlinks
plugin was taking the minimum between `.value.timestamp` and `timestamp`
but since we're not using SSB-Backlinks in this query (or maybe at all?)
we get to make our own little hacky query. Cel taught me that the
`$sort` option causes problems here because it buffers the entire result
in memory before doing the sort. Boo. Instead we just need to reference
the property that we want to be sorted by with some operation, even if
it does nothing (like `{ $gt: null }`), but since we're filtering time
travelers we actually need `{ $lge: Date.now() }` there anyway.
Problem: We load the default config with `require()`, which would allow
arbitrary JavaScript execution, are producing debug output everywhere
with `console.log()` statements, and don't have instructions for how to
write the config file. The config file is also `.config/**/config.json`,
and I think it would be helpful to specify that this is the **default**
config rather than any other kind of config.
Solution: Rename the config to `~/.config/oasis/default.json`, replace
the `require()` with `fs.readFile()`, remove `console.log()` for
debugging, and add some example JSON for how to use the config file.
This should give pretty much expected behaviour. Each config value
can be set by three sources:
1. By command-line argument. If it is not given, then
2. By config file. Or, lastly
3. By default value in the source code.
I can't test that the config file is searched and read from the right
place on windows or macOS, but on linux it works.
Problem: We're doing tons of unnecessary cryptography by encrypting the
connection between the "client" and "server", which are often running in
the same process.
Solution: Instead of connecting to the SSB service over TCP and
encrypting the stream, just connect over a socket (supported on Windows,
macOS, and Linux) and don't bother encrypting anything. This is what
Patchwork and Patchbay do already, and since our secret is at
`~/.ssb/secret` then we should be comfortable with `~/.ssb/socket` being
a trusted file where access implies authentication.
Local tests suggest that when sodium-native is available, this commit
reduces the time to render the 'Popular (Day)' page by 17%, but when we
have to fall back to JavaScript cryptography the same page now takes 30%
less time to render. My intuition is that this improvement is more
dramatic on mobile, but requires further testing before we can pat
ourselves on the back too much. :)
Problem: We've been using a workaround to filter the results of
SSB-Mentions, but that's just been fixed upstream and so we don't need
the workaround anymore.
Solution: Update the SSB-Mentions package and remove the workaround.
Problem: During the SSB-Query refactor I used `{ private: false }` in an
attempt to hide private messages, but apparently this option is only
available via SSB-Backlinks, so it was ignored without any warning (!).
This allowed private messages to be displayed in public views.
Solution: Remove the unused option and add a filter to public views that
ensures that private messages are always hidden.
Problem: Private messages should only be showing up in the Private page,
not the Mentions page, but right now they're showing up in Mentions.
Solution: Add a check to ensure that Mentions doesn't have any private
messages.
Problem: When the port was already taken by another process we've been
dumping a big esoteric error message that really isn't helpful for most
people.
Solution: Catch the error and give our users a friendly error message
with a suggestion on how to run Oasis on another port.
Problem: I added an image border to try to make it easier to see the
boundaries of images but it's not really necessary and at least one
person didn't love it, which is grounds for removal.
Solution: Get it out of here! This commit removes the image border and
also removes a random `console.log()` that shouldn't have been left in
`src/index.js`.
Problem: After publishing a message you were redirected to the Popular
view, which wasn't good feedback that your message had been published.
Solution: After publishing a new message, redirect to the Latest view.
Problem: It was impossible to tell which version of Oasis you're
running, which makes it hard to know when you need to update.
Solution: Put the version in the settings so it's easy to reference.
Problem: The `messagesByType()` queries don't let us sort by asserted
time, which means that we can get messages from a long time ago that
just ended up on our computer.
Solution: Use SSB-Query everywhere. The query syntax isn't really my
favorite but it lets us make declarative queries that respect the
asserted timestamp instead of just the received timestamp.
Problem: There wasn't an easy way to copy and paste a message ID, feed
ID, or blob ID and get directed to that page in Oasis.
Solution: When someone pastes one of the above into the search, just
redirect the browser to the correct page rather than searching for that
text.
Problem: Until we have a way to change the view of each page, it seems
to me that each of the views that we're experimenting with should be on
their own page rather than modifying the existing pages to add more
functionality. I'm also noticing that I can't process all of the
information that's on the screen when the new horizontal layout is used,
and I think we would benefit from using the previous layout
consistently.
Solution: Move the summary view to its own page so that we can continue
to iterate on it without modifying the popular view, which is already
complex, and revert the style changes to maintain consistency with the
rest of the linear design (threads go downward, indentation denotes
sub-threads, etc).
Problem: Recently we refactored the code so that we calls over MuxRPC
used promises, which was incompatible with the client-side SSB-Tangle
plugin that was expecting a callback. While debugging this error, I also
noticed that our `{ ws: { http: false } }` code wasn't doing anything,
and that our connection error handler was handling all kinds of
unrelated errors that would make debug more difficult.
Solution: Use `util.promisify()` to convert the callback-style function
into a promise-style function, remove the unused SSB-WS code, and reduce
the scope of the error catcher so that it'll continue to output errors
unless they're "cannot connect to sbot".
Problem: When we couldn't figure out someone's name or description we
were using `null`, which got coerced to a string and ended up with us
calling people `"@null"`. Not ideal!
Solution: Support a use-case where someone doesn't have a profile
description or a name, and use the first 8 characters of their public
key for their name if they haven't set a name themselves.
Problem: The new custom publish was called 'Manual mode', which I'm
afraid isn't super easy to understand for people who don't already know
SSB. The custom publishing page didn't follow the same conventions as
other pages, either.
Solution: Hide the custom publish mode behind the publish page, with a
warning that it's for 'advanced users', and ensure that it uses the same
conventions and visual styles as other pages and code around the app.
Problem: The new content warning code was applying a content warning to
all posts, which tried to publish an invalid content warning `""` and
threw an error.
Solution: Only publish a content warning if it's a string with non-zero
length.
Problem: Emoji from Markdown posts were being displayed in the default
font, which doesn't always use color emoji.
Solution: Use the `.emoji` CSS class on those emoji to ensure that the
correct emoji font is used.
Problem: When replying to a private thread we already know who the
recipients are, and they're already going to get the notification, so
there's really no reason to add a mention.
Solution: Remove the mention when the message is private.
Problem: Previously we were only counting likes from people you follow,
but showing messages from anyone. This was backward, and could
potentially show messages from blocked authors that were liked by people
you follow.
Solution: Move the `socialFilter()` invocation down the pipeline so that
it sorts the output messages, not the likes.
Problem: Any error thrown while looking for thread ancestors was
throwing a "message not found" error, which was incorrect and useless.
This error caused me to his refresh repeatedly, not understanding that I
was publishing multiple messages. Super bad. While investigating this I
found that there's a slightly different problem where someone can post
an *invalid* message link, which we don't currently have handling for.
Solution: Only show that error when it's actually happening, and add
support for just ignoring when we see an invalid message link as `root`
or `fork`.
Problem: In the old `cooler.read()` implementation, sources returned
promises, but that isn't the case in the native SSB-Client promise
impelementation. This means that in at least one place, there's a bug
where it tries to call `.then()` on a non-promise.
Solution: Remove promise-based code from streams, which don't require
any special handling anymore.
Problem: Your posts show up in Extended, which is unexpected because I'm
the center of my network, not some rando at the periphery.
Solution: Use the `socialFilter()` function to make sure that the
extended view only shows people in your extended network, not you. :)
Problem: We started using `cooler.get()` and `cooler.read()` because it
was impossible to use promises with SSB-Client.
Solution: I made some downstream pull requests into the MuxRPC module
and the SSB-Client module, which means that both of them now natively
support promises. This commit removes the weird convenience methods and
replaces them with the native promise support, which should hopefully
make the code easier to read and write.
Problem: There was one missing component that would filter out nicknames
from other people. This caused a problem where we could get a name for a
feed but it could've been assigned by a friend, which we don't want
right now.
Solution: Ensure that the subject of the message is the same as the
author of the message.
Problem: Search engines are controversial and my understanding is that
most people on SSB don't want their messages indexed by search engines.
If that's the case, we should probably disable it.
Solution: Add basic `robots.txt` file to ask search engines to stay away
and please don't save info. I'm concerned that, like `publicWebHosting`
redactions, it gives a false sense of privacy, but it seems like this is
probably what most people would want?
Problem: The SSB-About plugin is incompatible with our needs. More info
in the GitHub issue linked below and in the code comments.
Solution: Unfortunately, roll our own alternative to the SSB-About
plugin so that we can be 100% sure that it pulls the latest 'about'
published by an author about themselves and doesn't just skip `false`.
Problem: The `socialFilter()` function wasn't documented and contained a
bug where it wouldn't show your posts when `following = true`. This is
because you usually don't follow yourself, so `following = true` was
basically equivalent to `me = false`.
Solution: Add some documentation and resolve the bug by adding special
handling for when the message is from us *before* passing to the general
implementation for follow/block checking.
Resolves https://github.com/fraction/oasis/issues/155
Resolves https://github.com/fraction/oasis/issues/177
Problem: It's hard to show off Oasis or take screenshots without
respecting the `publicWebHosting` convention. While `publicWebHosting`
lacks a formal specification and I'm a bit confused about what its
boundaries are, it sounds like some of our friends would like to avoid
us publishing any of their content on the public web if we can avoid it.
Solution: Add --public option that turns Oasis into a public web viewer.
This makes it **slightly inconvenient** to see these public posts, but
should absolutely not be mistaken for a privacy guarantee. Only HTTP GET
endpoints are allowed, so random people can't publish or change
settings. The name, avatar, description, content warning, and message
contents are replaced with "Redacted", but again, this is all public
information that we can never provide real privacy for.
Resolves https://github.com/fraction/oasis/issues/48
Problem: We use nosniff to keep the web browser from getting confused
about what kinds of content we're serving in Oasis, but this causes
problems for blob URLs that have arbitrary data.
Solution: Remove nosniff on blob URLs to let the browser figure out what
kind of content we're serving.
Resolves https://github.com/fraction/oasis/issues/138
Problem: The regular space in the sidebar emoji was breaking the line at
a specific viewport width and the emoji were being shown as the wrong
font.
Solution: Use a non-breaking space and `font-family: initial` for
full-color emoji instead of using the system font.
Resolves https://github.com/fraction/oasis/issues/150
Resolves https://github.com/fraction/oasis/issues/153
Problem: The socialFilter was hiding posts published by the user, which
felt weird and uncanny.
Solution: Fix the default so that `{ me }` isn't hidden from a view
unless the model specifically wants that to happen.
Resolves https://github.com/fraction/oasis/issues/156
Problem: The Private and Mentions page didn't have view labels yet, and
Publish should be its own page instead of being at the top of every
page.
Solution: Inspired by @cinnamon-bun's work to add friendly view labels,
plus a new Publish page. This also moves the period selection from the
popular page into the view label, which felt better to me with the
previous `<section>` background. I also tried a different text format
for describing the pages, using a common form and using `<strong>` to
draw attention to any change from "Posts from people you follow, sorted
by recency" which feels like the expected default for most people.
@cinnamon-bun: To me this feels like a fun back-and-forth where I'm
riffing on your work and hoping that you do the same, but if it feels
wrong/rude please let me know. Trying to work on designs with a system
like C4 is super new for me and I wouldn't be surprised if there are
pain points to fix!
Resolves https://github.com/fraction/oasis/issues/160
Problem: There was no way to onboard new users since we couldn't redeem
invites.
Solution: Add basic follow-back invites to the settings page. This takes
an invite string, runs it through invite.accept, and either returns the
error in full *or* redeems the invite quietly.
Problem: @masukomi pointed out that the 'latest' view doesn't show the
awesome slice of content that you'll see if you just look at root posts.
Solution: Let's experiment with them! This commit adds a 'Topics' page
that has the latest root posts from people you're following.
Problem: The nav is aligned to the left of the screen on mobile, which
feels off-center and unbalanced on mobile. https://github.com/fraction/oasis/issues/135
Solution: Center the menu to optimize for space around the links.
Problem: We shouldn't be showing any strangers on the popular or latest
pages. We shouldn't be showing anyone who's been blocked on any of those
feeds.
Solution: Don't show blocked people on any pages and don't show
strangers on the popular / latest pages.
Problem: If the user tries to see a thread and the link points to a
message we don't have, then we don't have any way to display anything in
the thread. How could we even know which thread it's in?
Solution: Throw the error but make it more useful and fix the "non-error
thrown" verbiage that we've inherited from a dependency trying to throw
a non-error.
Problem: I think during a refactor this code was changed and ended up
breaking the "fake image" that we return when the user doesn't have an
image. We also don't see image errors because they aren't in the browser
viewport if they return text and we don't `console.error()` our errors.
Solution: Fix the image code to return a PNG as a buffer and duplicate
errors to stderr.
Problem: SSB-Search has a bug where too few characters just hangs the
search indefinitely and never returns. https://github.com/fraction/oasis/issues/107
Solution: Enforce a minimum length of 3 characters in the search. I
bumped into another bug where HyperScript, a dependency of HyperAxe,
doesn't support the `minlength` attribute, so I had to deploy a small
workaround for that too. The fixes aren't very pretty but they're better
than just ignoring the problem.
Problem: The previous commits didn't persist language choices and the
drop-down's initial value wasn't respecting the language you selected.
Solution: Persist the language choice in a cookie, defaulting to
English, and build the drop-down with the selected language. This also
changes the word "Spanish" to "Español", and slightly refactors
`http.js` to accept *middleware* rather than just routes. This lets us
add other middleware, such as the language selection middleware added in
this commit.
Problem: The previous commit added English as a language but didn't add
other languages or ways to switch between them.
Solution: Add the most primitive language selection possible and a few
small translations contributed by @bramdroid during a totally unrelated
conversation. This does not persist the language selection and doesn't
auto-select the current language from the dropdown, but those should be
easy to add in the future.
Problem: It was impossible to do any internationalization because
strings were all embedded throughout Oasis.
Solution: Add an internationalization submodule that provides
language-specific strings for the text elements in views. In future
commits we can add language selection and fallbacks for when the
selected language doesn't support the text we need to have translated.
Problem: The 'Following' page was super slow because we were doing a
MuxRPC request on every single message.
Solution: Do one request to see who we have relationships with, filter
out the people we aren't following, and then check against that list
instead of doing a bunch of MuxRPC calls.
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `curl http://localhost:3000/public/latest/following` | 500.9 ± 163.4 | 392.1 | 881.4 | 1.00 |
| `curl http://localhost:3456/public/latest/following` | 4663.7 ± 184.6 | 4438.6 | 5075.2 | 9.31 ± 3.06 |
