Updates to the Groups.io service.

Subscribe via RSS

---

June 12, 2020

• UPDATE: Updated members manual with changes from Nina.
• CHANGE: Made our email parsing code a little more liberal so that it wouldn't reject mime parts with content-type lines containing duplicate `name` parameters.
• BUGFIX: For really tall browser windows, infinite scroll would sometimes not trigger.
• BUGFIX: Database search results were missing multiple choice row answers.
• CHANGE: Expire any muted hashtags when viewing the hashtags page.
• BUGFIX: In the Week and Day views of the calendar, when clicking a slot after noon to create an event, we would not parse the time correctly.
• API: Added `default_hashtag_view` field to the user object, which can be set using the `/updateuser` endpoint.
• CHANGE: Store and load FuelUX package locally instead of relying on the //fuelcdn service.
• CHANGE: In the edit pending message page, include hashtag descriptions in the `Add Tags` selector.
• CHANGE: Display number of topics in Hashtags list view.
• CHANGE: We are now only including the first two attachments for every message in a digest, to prevent oversized digests.
• CHANGE: Images in digests are now resized using a JPEG quality of 75.
• CHANGE: Change the order of evaluation so that a member set to MT is evaluated before a group's moderate first message setting.
• CHANGE: Move the Group Email Addresses block on the Group home page.
• CHANGE: Update the /mutehashtag URLs in message footers.

June 5, 2020

• CHANGE: Updates to the members/owners manuals from Nina.
• CHANGE: Photo thumbnails, including the single photo view, are now resized to 2048 on their longest dimension, as opposed to 720 previously. The JPEG quality was also decreased to 90. This only applies to photos uploaded from now on.
• INTERNAL: Re-wrote the program that sent daily summaries to make it much more efficient; the old version would sometimes take more than an hour to complete a run (and is run every hour). This was causing some people to miss summaries, especially those with America/Chicago timezones.
• INTERNAL: Fix issue with the GoAMZ MultiDel() call not working on S3 to remove files and photos when deleting a group.
• NEW: Hashtag list view shows mute/follow duration. Also, redid the mute/follow hashtag screens.
• CHANGE: For new donation requests, we have changed the fee structure: the Groups.io fee is 2%, in addition to Stripe's credit card fees.
• CHANGE: Display the default 'Elephant' group icon on the mobile groups page for groups without an icon set (replacing the group/pie chart icons).
• SYSADMIN: Upgraded the TLS certificates for the *.groups.io domains to use Let's Encrypt certs instead of the older Gandi issued cert, to fix the Safari cert warning.

May 29, 2020

• INTERNAL: Work towards supporting notifications.
• BUGFIX: The timezone listed when viewing an event or in the calendar popup sometimes did not match the text when creating the event (the UTC offset was the same, just the city names could be different, but correct).
• SYSADMIN: Prep work for adding a second email server.
• INTERNAL: Sped up deletion of photos and files when deleting a group.
• CHANGE: Formatting changes in the subgroup listing page.
• NEW: You can now toggle between the existing gallery view and a new list view in the Hashtags page.
• INTERNAL: Re-wrote several web templates to address some technical debt issues.

May 22, 2020

• CHANGE: Updates to the Members manual and Owners manual from Nina.
• SYSADMIN: Upgraded the kernel of mail01, to hopefully address slowness issue.
• CHANGE: When a group is set to Reply To Sender and also has Remove Other Reply Options set, do not add a List-Post header to individual messages.
• BUGFIX: Daily summaries included topics that had been muted.
• NEW: Remember the current calendar view.
• NEW: Revamped mobile Groups page.
• CHANGE: Made the subject lines of the subscribe/unsubscribe emails more useful.
• API: Added `/getenterpriseinfo` endpoint.
• BUGFIX: The subject wasn't logged in the activity log when deleting or rejecting a pending message.
• CHANGE: Relaxed the XML parsing strictness of RSS feeds to allow feeds that would otherwise be considered broken.

May 15, 2020

• BUGFIX: We were linking to incorrect messages in the activity log when someone was removed for marking a non-message as spam.
• CHANGE: Updates to the Members Manual.
• CHANGE: Unbolded all labels on the website and unbolded all panel titles.
• CHANGE: Various wording tweaks on the website to make things more consistent.
• BUGFIX: Fix Zoom meeting integration behavior when the Oauth token expires.
• BUGFIX: Generate a longer password when setting up a Zoom meeting.
• BUGFIX: Fix redirect when asked to log in to a group that has a subdomain.

May 8, 2020

• CHANGE: Allow `background` and `mso-highlight` CSS styles in messages.
• CHANGE: Various icon changes for more consistency.
• CHANGE: Updated the Members Manual with changes on Wiki page URLs.
• NEW: When a person is banned from Groups.io, we now log that in all groups they were a member of, and we remove all subscription data instead of just hiding it, which should keep sub counts correct.
• BUGFIX: The sending of a new confirmation email by a moderator using the 'Send Confirmation Email' button was not logged properly in the email delivery section.
• NEW: Logging to the activity log when we send the group guidelines out and when we send monthly reminder messages out.
• INTERNAL: Optimized a database call when generating digests.
• BUGFIX: Don't include links to profiles of banned members when viewing messages.
• CHANGE: When visiting a manual in the help center, display the table of contents.
• CHANGE: Wiki pages now use the wiki page ID instead of the title (with exceptions for _Sidebar/_Footer) to prevent link breakage when titles change.
• INTERNAL: A lot of work to enable distributed tracing of the internal systems.

May 1, 2020

• NEW: Updated Help Center manuals with Nina's updates, including reformatted PDFs.
• SYSADMIN: Continued debugging issues with the new database server.
• CHANGE: Documented photo/file upload size limits and increased them for premium/enterprise groups.
• SYSADMIN: Upgraded the web servers with more memory to deal with increased file/photo upload sizes.
• CHANGE: Rolled back the List-Id change from 4/25/20.
• BUGFIX: Sorting by file type in the Files section would sometimes return the same results over multiple pages because our sort criteria wasn't stable.
• INTERNAL: We now clean up unused RSS feeds in the database.
• INTERNAL: On an edited message, check if the digest message exists already before adding it, to prevent postgres log messages.
• INTERNAL: Changed nginx log format back to combined to goaccess can parse user agents.
• INTERNAL: Work to make it easier to add additional FBL providers.
• API: `/getevent` wasn't obeying the group calendar permissions setting.
• NEW: We now parse more information about which message was flagged as spam from an FBL report, and send that out to the user and group owners.
• CHANGE: The List-Id email header now includes the group id, to better help us parse FBL reports.

April 24, 2020

• CHANGE: We now automatically delete drafts that are older than two weeks.
• INTERNAL: We now reuse SMTP connections when sending email to major providers, instead of opening a new connection for each email.
• INTERNAL: On advice from Comcast, subscribed to the Comcast FBL, along with several other FBLs.
• SYSADMIN: Upgraded the kernel in the main database machine to a more recent version.
• INTERNAL: Added grafana annotations to all cronjobs.
• SYSADMIN: Reduced comcast.net karld concurrency from 10 to 5.
• INTERNAL: Improved our email monitoring dashboards to better track when providers are not accepting email from us.
• INTERNAL: Optimized forcesummaries some more (and again fixed its use of replicas).
• INTERNAL: Use db replicas for wiki queries.
• INTERNAL: Use db replicas for some auth queries.
• SYSADMIN: Upgraded the kernel in db replica machines to a more recent version.
• SYSADMIN: Changed when we run the stats process on db05.
• SYSADMIN: Added pg_stats_activity counts for db04/db05.
• SYSADMIN: Reduced comcast.net karld concurrency from 20 to 10.
• SYSADMIN: Reduced cox.net karld concurrency from 10 to 5.
• INTERNAL: Optimized the replacement of email addresses in messages with links to user profiles.
• INTERNAL: Fixed some database indices.
• SYSADMIN: Adjusted main database parameters to deal with the current load.
• SYSADMIN: Moved pgbouncer off main database machine to its own dedicated instance.
• INTERNAL: Changed the forcesummaries process to use database replicas.
• INTERNAL: Added additional stats graphing to the internal monitoring dashboard.

April 17, 2020

• INTERNAL: Changed how we were limiting email sent to some domains, to use the MX server domain as a key for limiting instead of the receipient's domain. This will affect all messages to GoDaddy hosted domains, orange.fr/wanadoo.fr, earthlink/mindspring/netcom and a couple others. This will hopefully allow us to keep under per-connection limits imposed by these email service providers and avoid getting temporarily blocked for trying to send too much email at one time.
• INTERNAL: Technical debt cleanup of web page templates. An ongoing process.
• CHANGE: Updated the Help Center manuals with updates from Nina.
• API: `/downloadarchives` was not returning the messages in ascending date order.
• CHANGE: Allow 'list-style-type' styling in user generated content.
• CHANGE: Some formatting tweaks to the Member page, including displaying the member's timezone.
• CHANGE: Changed 'Unsubscribe' button in the Subscription page to 'Leave Group' to be more clear.
• SYSADMIN: Switched primary database machines because of continuing (minor) corruption issues.
• CHANGE: When viewing a member's information on mobile, changed the nav buttons at the top to a dropdown instead. Also fixed several navigation bugs when viewing past members.

April 10, 2020

• INTERNAL: Work to investigate which technologies to use for the app.
• BUGFIX: Misc improvements to formatting of the mobile website.
• NEW: (Re)launched the Zoom calendar integration.
• BUGFIX: The left chevron icon in the photo view was not vertically aligned correctly.
• BUGFIX: Paging backwards in the Emailed Photos photo viewer stopped working if you paged back at the first photo.
• API: Fixed pagination bugs with `/getevents` endpoint.
• CHANGE: Switchover to the new Help Center, with finalized Owners and Members manuals and FAQ.
• CHANGE: Adjusted the size and alignment of the lines separating a group's footer from the rest of the message footer information.
• BUGFIX: When creating a new plain text wiki page, the page would internally be marked as being HTML, so on future edits the HTML editor would be used.

April 3, 2020

• INTERNAL: Improvements to our log file grepping system to help track down problems.
• NEW: Many updates to the new Help Center, Owners Manual and Members Manual.
• CHANGE: Lots of little wording tweaks on the website to make things more consistent.
• CHANGE: For messages sent to the +owner address, delete any existing To header and replace it with listname+owner@groups.io, to make it more clear where the message was sent.
• API: Added `/newgroupalias` and `/deletegroupalias` endpoints.
• NEW: Added a search bar to the new group home page/did some rearraigning of the page (as seen on beta@).
• API: Added `/unbanmember` endpoint.
• BUGFIX: When uploading a directory to the Files section, the description was not added to the top level folder.
• BUGFIX: Repost was not working in the Topic view.
• NEW: Added Zoom integration to the calendar. You can now set up Zoom meetings when you create new events.
• CHANGE: Cap message subject line length at 1000 characters.
• CHANGE: For groups set to Reply To Sender, no longer add the CC header to point back to the group. When replying-to-all, it would create a duplicate mailing.
• CHANGE: Improved the formatting of the new Owners Manual PDF.
• SYSADMIN: Upgraded the machine instance responsible for sending email.
• INTERNAL: To display attachments on the website, now redirect to S3 in many cases instead of us fetching/returning the data.

March 27, 2020

• INTERNAL: Work on new Zoom integration with the calendar.
• CHANGE: Do not allow setting up of two factor authentication when you are logged in using Google or Facebook.
• API: Changing a subgroup's name using the `/updategroup` endpoint would fail.
• API: Removed `bcc_me` parameter from `/postdraft` as that was unused.
• BUGFIX: Private group guidelines were accessible to non-subscribers via the +guidelines email command.
• INTERNAL: More logging to track email delivery speed
• INTERNAL: Optimizations for lowering the time to deliver email
• BUGFIX: Removed do-nothing `Edit Settings` menu option when viewing moderators on the Members page.
• BUGFIX: ICS files containing yearly repeating events were not parsable.
• API: `/newdraft` endpoint now supports `draft_type_post_to_sub` draft_type.

March 20, 2020

• INTERNAL: Work on new Help Center and code to convert the new manuals from Google Docs to HTML.
• CHANGE: In the database table page, replaced the individual visibility toggle buttons with a dropdown menu.
• NEW: Database address columns have a default country.
• CHANGE: Changed width of all selects on the website so that they are their natural width, not 100%, which was confusing to some and ugly to all.
• CHANGE: For Multiple Choice, Multiple Select columns, use a series of checkboxes instead of a HTML select element.
• CHANGE: Improved look of the database add row page.
• NEW: Database columns have a new `description` field.
• API: Added `/getactivitylog` endpoint.
• API: The docs incorrectly left off the `/get` prefix for the `subgroupcategories` endpoints.
• API: Passing in the `type` parameter to `/getmembernotices` would return a blank response.
• BUGFIX: The database map wasn't working for Enterprise domains.
• BUGFIX: If an address field in a database was set to a non-default color, the color would not show up in the add/edit row screen, potentially causing the labels to also disappear.

March 13, 2020

• INTERNAL: Work on a new help center.
• BUGFIX: If an event had '<<' in the name, the group event notice would not be generated correctly.
• BUGFIX: Export user wasn't generating ICS files correctly.
• API: Added `email_address` field to `group` object.
• BUGFIX: There was a bug with event RSVPs which could prevent some people from RSVPing to an event.
• BUGFIX: Removed duplicate `Member Sync` buttons from various settings pages. This had been renamed to `Service Sync` and placed elsewhere.
• API: Fixed sort for `/getmembernotices` endpoint.
• CHANGE: Separate the settings in the `Default Sub Settings` page into two panels for greater clarity.
• CHANGE: For moderator only hashtags, including Special hashtags, we now allow their use if they've been added to an approved pending message sent from a non-moderator. This means they were added by the approving moderator before the message was approved.

March 6, 2020

• API: Most endpoints now support the `extended=true` parameter.
• API: Endpoints `/unbounceuser`, `/deleteprofilephoto`, `/resendconfirmationemail` now require a POST request along with `csrf` token.
• API: Added additional documentation around `extended=true` parameter.
• API: Added `token` parameter to `/login` endpoint to allow generation/login using HTTP Auth instead of a cookie.
• CHANGE: When creating a group, if the user has a timezone set, we use that for the default timezone of the group.
• BUGFIX: In some cases where a moderator claimed a pending message and then left a group, an erroneous claiming badge would be displayed on their previously claimed pending messages.
• API: Added `bouncing` option for `type` parameter of `/getmembers` endpoint.
• BUGFIX: When your default message sort is newest first, and you click on the oldest month on the group home page, previously it would not go to the first message in the archives as expected
• INTERNAL: Updated the Let's Encrypt Cert renewal process.
• BUGFIX: The Github add integration flow did not work with enterprise groups. Also, the Github API is being updated and we needed to upgrade the library we use to access it.
• CHANGE: Changed more instances of `thread` in the UI to `topic`.
• INTERNAL: Optimize forcedigests process.
• BUGFIX: In generated ICS files, we were HTML encoding the Organizer field when we should not have. This was causing parsing errors with some calendars.

February 28, 2020

• CHANGE: Previously, we would force a digest to be created at 10:30pm Pacific Time nightly, if one had not been generated in the previous 6 hours and messages had been sent to the group since the last digest. Now, we force a digest to be created at 6AM local time for each subscriber, based on their timezone, if a digest had not been generated in the previous 6 hours and messages had been sent to the group since the last digest.
• CHANGE: When creating a new group or changing the name of an existing group, change the error message when the group name is already taken to `That address is not available.`.
• CHANGE: Hardcode some IP addresses associated with chinamobile.com because reverse DNS lookups on them fail randomly.
• BUGFIX: The command to update whether a topic was locked/moderated/etc in the search index was failing because the syntax changed between Elasticsearch versions. Fixed the command and re-indexed the message archive.
• BUGFIX: When joining a new group, the membership parameters could not be modified from that page.
• INTERNAL: Upgraded to Go 1.14 compiler.
• NEW: Database columns now have a `Default Hidden` checkbox.
• NEW: Database tables now have `ID Default Hidden` and `Updated Default Hidden` checkboxes.
• API: Added `default_hidden` field to the `Database Column` object.
• API: Added `id_default_hidden` and `updated_default_hidden` fields to the `Database Table` object.
• BUGFIX: Wrong logo used for Google Drive.
• API: Removed the `invalid posterid` error from `/searcharchives` to allow people to search for messages from people who are no longer members of the group.
• API: Added `body` field to `message_search_result` object.
• NEW: Member Directory Search.
• API: Added `/searchmemberdirectory` endpoint.
• BUGFIX: When deleting attachments to make space, we were not taking into account space used by files and photos, so we ended up deleting fewer attachments than we should have.
• BUGFIX: Fixed pagination when searching on members.
• SYSADMIN: Added an additional hot standby instance of the main database server.
• CHANGE: In the UI, changed `hash tags` to `hashtags` for consistency.

February 21, 2020

• NEW: Member search now also searches moderator notes.
• BUGFIX: In some cases when we do a reverse DNS lookup on the IP address of a server sending us email, the Go DNS library would return a `server misbehaving` error, which we treated the same as not having a reverse DNS record, causing us to reject the incoming message. In those cases, we now repeat the lookup using an alternate DNS library. Specifically, China Mobile email servers seem to suffer from this.
• BUGFIX: For some queries, wiki search would return a blank second page.
• BUGFIX: For search queries with URL encoded characters, we would not properly decode them correctly on the subsequent search pages.
• API: Misc improvements to the documentation.
• API: Added "bad_request": "no space" errors to `/addphotos`, `/updatephoto` and `/addfiles` endpoints.
• CHANGE: Moderators now cannot change the role of Owners.
• CHANGE: Moderators now cannot change the subscription options of Owners.
• API: Added support for `extended=true` to the drafts API endpoints.
• API: `/updatemember` can now return: "bad_request": "moderators cannot change role of owners" is returned if a non-owner attempts to change the `mod_status` field of an owner.
• API: `/updatemember` can now return: "bad_request": "moderators cannot make members owners" is returned if a non-owner attempts to make a member an owner.
• API: `/updatemember` can now return: "bad_request": "moderators cannot change owner subscription settings" is returned if a non-owner attempts to change an owner's subscription settings.

February 14, 2020

• INTERNAL: Added additional monitoring of the search cluster.
• INTERNAL: Switched over to a new Elasticsearch 7 search cluster. All of this work around search has multiple goals:
• INTERNAL: Re-indexed all of our data in a new search cluster.
• BUGFIX: Found a file descriptor leak in some networking code.
• INTERNAL: Began deleting attachments in groups that are over their storage allotment.
• CHANGE: Changed more instances of `subscriber` to `member` on the website.

February 7, 2020

• CHANGE: Standardized on the term `member`; we used `subscriber` and `user` previously. Several tweaks to the group settings page. Changed NuM to NMM.
• INTERNAL: Much work to support moving from Elasticsearch 5 to Elasticsearch 7, including optimizations for full reindexing.
• BUGFIX: Introduced and then fixed a bug that caused errors when trying to join a subgroup.
• CHANGE: The text when creating a group said that new members would be NuM 2, but the group was created with NuM 1. All new groups are now created with NuM 2.
• API: Removed `sub_group_access` from `/creategroup` endpoint.
• CHANGE: Removed the Create Subgroups dropdown when creating a new group since subgroups are now a premium feature.
• INTERNAL: Switched from one SSL certificate for all enterprise domains, to individual certs for each domain.
• BUGFIX: Changed how we sort messages in the Messages/Expanded messages view to use message created date instead of object date, to account for groups that had messages imported that were not strictly date ordered (or that already had messages in the archives).
• NEW: Rate limiting joining groups and other things to help detect/prevent the `mass subscription` attacks we were seeing this week.

January 31, 2020

• INTERNAL: Better tracking of which mechanism a user is added to the system, to aid in spotting shenanigans.
• SYSADMIN: Added a lot of storage to increase the amount of historical logs we can keep.
• INTERNAL: Work on converting to elasticsearch 7.
• INTERNAL: Optimizations to the change data system that ingests data into the elasticsearch cluster. We were replaying a lot of old LSNs after each restart that we didn't need to.
• CHANGE: In the email announcing a pending subscription, change `rejected` to `deleted` to explain what happens after 14 days.
• SYSADMIN: Added an additional 6 nodes to the search cluster because searches were taking too long.
• BUGFIX: After searching/finding a photo, the Download button did nothing.
• BUGFIX: Viewing the Emailed Photos album using any sort order other than Posted would lead to the wrong order when viewing individual images.
• CHANGE: For Enterprise SSO, first redirect through the groups.io domain, then to the SSO provider, like we do with Facebook/Google.
• API: Added `start_time` parameter to `/downloadarchives` endpoint.
• API: Updated docs on `/updategroup` endpoint, adding some missing parameters.
• NEW: New premium feature, Send Invites On Join, sends an ICS file of all calendar invites when someone joins a group, and sends an ICS file of cancel notices when someone leaves a group.
• BUGFIX: Re-write of how we were generating calendar ICS files to clean up some technical debt but more importantly to generate more correct files in regards to repeating events. Previously the ICS files just contained single events for each instance of a repeating event instead of specifying the repeating event itself. This led to issues when cancelling events, both via ICS files as well as when someone tried to delete an event in their (third party) calendar.
• BUGFIX: Flash notices (like 'You are not allowed to create any more groups today') were not displayed in some instances when your home page was /groups.
• CHANGE: A group that is not restricted can no longer have `Members Visible` set to `Subscribers`, unless it is a subgroup and the parent group is restricted. This is prevent email address harvesting.