Changes to the site this week:

• CHANGE: When emailing RSVP responders, default the From line to the person sending it (it would normally default to +owner if a mod was sending the message).
• CHANGE: Exporting event RSVP responses now returns a CSV file, containing all responses.
• DOCS: Updates from Nina and beta of the new Getting Started Guide for Owners.
• NEW: Enterprise groups can set the logo used for web push notifications.
• NEW: When you RSVP yes to an event (and are not waitlisted), you will get an email notification.
• CHANGE: Only display RSVP additional information fields for moderators/event owners.
• CHANGE: Consolidated some RSVP notification messages.
• NEW: When a moderator/event owner changes an RSVP, a notification is sent to the person.
• CHANGE: Changed help wording of RSVP yes message when creating an event.
• CHANGE: Locking a topic that contains a poll now also now closes the poll.
• INTERNAL: Optimized the code for converting an HTML string to plain text to deal with some pathologically large style blocks we encountered in an uploaded file, which was causing exceptionally long runtimes and timeouts.
• CHANGE: Changed terminology from maybe to not sure for event RSVPs.
• BUGFIX: Don't send out a yes message email to someone RSVPing to an event if there is no yes message.
• NEW: You can create files in the Files section that are links to Google Docs/Dropbox/Box/OneDrive documents.
• BUGFIX: Fixed links in Files search results.

Take care everyone.

Mark

Changes to the site this week:

• BUGFIX: Fixed sorting of Files section search results.
• INTERNAL: More work converting templates to quicktemplate.
• UPDATE: Owners and Members manuals updates from Nina.
• SYSADMIN: Upgraded the machine instance dedicated to virus scanning.
• NEW: If your position on an event waitlist changes, you will be notified by email.
• NEW: You can select either a grid or list display when viewing event rsvps.
• NEW: Moderators and the event owner can now manage RSVPs, changing them to Attending/Not Attending/Not Sure. These changes are logged in the activity log.
• CHANGE: If you are on the waitlist for an event, change your RSVP to no or not sure, and then change back to attending, you're put at the back of the waitlist.
• NEW: When someone is moved off the waitlist, it is logged in the activity log.
• BUGFIX: Fixed several issues with ICS file generation, especially around UTF-8 conversion. Also stopped stripping out ':' characters, because the other fixes addressed the original issue with ':'.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Continued converting web and email templates from Go's template engine to quicktemplate.
• UPDATE: Owners Manual updates from Nina.
• NEW: When creating a group, you now have the option of a 30 day free trial of (almost all) premium features.
• NEW: Added additional Why Email Groups? marketing page.
• NEW: Activity log entries when a member changes their email delivery options.
• NEW: In the activity log, log specifics about moderators changing delivery and post moderation.
• CHANGE: Don't send a push notification to the moderator who approved/removed a subscriber.
• NEW: Added new activity log entries for adding/deleting member notices and also linked to the member notice being added/edited.
• BUGFIX: Fixed group notification of folder upload and fixed some instances of the word dir being used when it should be folder.

Take care everyone.

Mark

Changes to the site this week:

• UPDATE: Documentation updates from Nina.
• NEW: Monthly reminder notices can now be marked to be sent as special messages.
• NEW: Added missing Notify Members checkboxes to remaining file, photo and album actions with the exception of moving files (TBD).
• NEW: Added moderator notifications and Notify Member checkboxes for adding, modifying and deleting photo albums.
• NEW: When moving a photo, added a moved photo activity log entry, Notify Members checkbox, and moderator notification.
• CHANGE: In the Activity Log, change the word dir to folder when referring to Files Sections actions to be more consistent.
• NEW: When uploading photos, there is now a text box to enter a description.
• NEW: Added a Notify Members checkbox to the edit photo page.
• BUGFIX: We were not properly escaping commas, semicolons and slashes in DESCRIPTION fields when generating calendar ICS files. Also, Google Calendar has a bug where it will ignore the entire DESCRIPTION field if there is a colon in the text, so we now replace any colons with spaces.
• API: Added /getorg endpoint.
• CHANGE: For #guidelines-notice and #monthly-notice hashtags, sync the hashtag special field to the member notice special field every month.
• NEW: Monthly reminder notices can now be marked to be sent as special messages.
• NEW: When sending out #monthly-notice messages, mimic what we do with #guidelines-notice message, create the #monthly-notice hashtag with a delete timeframe of a month, and all mods options checked.
• CHANGE: In addition to not sending moderator push notifications to moderators that generated them, also don't send email notifications as well.
• BUGFIX: Changed z-index of floating table header to be below the Your Groups dropdown.
• BUGFIX: In the new moderator notifications, we were using the person's account display name, not the individual group display name (which is generally, but not always the same).
• NEW: Include more information in album and photo group export data.
• CHANGE: In digests, we were being too aggressive in stripping quoted sections of messages out when we knew the original message was not a reply. This led to some forwarded messages in digests missing key parts.
• BUGFIX: Summaries were being sent to pending members.
• API: Changed /registerdevice endpoint to /addpushsub
• API: Added /getpushsubs, /deletepushsub, and /deleteallpushsubs endpoints.
• BUGFIX: Fix reflow issue in database table view with new floatThead.

Take care everyone.

Mark

Changes to the site this week:

• CHANGE: Various moderator email and push notification cleanups.
• NEW: When viewing a database table, pin the header at the top of the page.
• BUGFIX: When deleting a database row from the /table view, we did not log that in the activity log.
• NEW: New Moderator Notifications for changes to Chats/Photos/Files/Wikis/Databases. Currently in testing.

Note: I will be on vacation next week, although I plan on enabling the new moderator notifications for everyone on Monday if there are no issues found during testing over the weekend.

Take care everyone.

Mark

Changes to the site this week:

• CHANGE: Guidelines sent monthly will now have the hashtag #guidelines-notice.
• CHANGE: Monthly notices will now have the hashtag #monthly-notice.
• BUGFIX: Notification messages did not obey group moderation settings.
• NEW: Added notifications for new photos uploaded and wiki additions/changes. Added the #file-notify hashtag to file upload notifications.
• CHANGE: When uploading files, the checkbox to notify the group is now checked by default.
• INTERNAL: Changed combinerd daemon to use nsqd.
• INTERNAL: Added a utility to check for and fix timezone subscription copy-down inconsistencies in the database.
• CHANGE: Fixed plain text digest subject lines to include subject tags/group titles
• NEW: Add a Send A Test Notification button in the Account notifications page to test if notifications are working.
• BUGFIX: Fix issue with merging a topic into a topic that had an apostrophe in the subject line.
• BUGFIX: If attempting to upload a photo to the Files section, and the group is configured to disallow that, we would still generate an upload notification.
• NEW: Web push notifications are now supported for Chrome, Firefox and Opera browsers via a new tab in the Account section.
• NEW: You can specify to have push notifications sent on a per-hashtag basis.
• NEW: Moderator notifications will now be sent by default via both email and web push notifications, if you have them configured. Moderator notification settings are now dropdowns instead of checkboxes.
• NEW: Moderator notifications for pending members and pending messages will only be sent to moderators with privileges to approve them.

Take care everyone.

Mark

Changes to the site this week:

• UPDATE: Updated members manual with changes from Nina.
• CHANGE: Made our email parsing code a little more liberal so that it wouldn't reject mime parts with content-type lines containing duplicate name parameters.
• BUGFIX: For really tall browser windows, infinite scroll would sometimes not trigger.
• BUGFIX: Database search results were missing multiple choice row answers.
• CHANGE: Expire any muted hashtags when viewing the hashtags page.
• BUGFIX: In the Week and Day views of the calendar, when clicking a slot after noon to create an event, we would not parse the time correctly.
• API: Added default_hashtag_view field to the user object, which can be set using the /updateuser endpoint.
• CHANGE: Store and load FuelUX package locally instead of relying on the //fuelcdn service.
• CHANGE: In the edit pending message page, include hashtag descriptions in the Add Tags selector.
• CHANGE: Display number of topics in Hashtags list view.
• CHANGE: We are now only including the first two attachments for every message in a digest, to prevent oversized digests.
• CHANGE: Images in digests are now resized using a JPEG quality of 75.
• CHANGE: Change the order of evaluation so that a member set to MT is evaluated before a group's moderate first message setting.
• CHANGE: Move the Group Email Addresses block on the Group home page.
• CHANGE: Update the /mutehashtag URLs in message footers.

Take care everyone.

Mark

Changes to the site this week:

• CHANGE: Updates to the members/owners manuals from Nina.
• CHANGE: Photo thumbnails, including the single photo view, are now resized to 2048 on their longest dimension, as opposed to 720 previously. The JPEG quality was also decreased to 90. This only applies to photos uploaded from now on. 6/4/20:
• INTERNAL: Re-wrote the program that sent daily summaries to make it much more efficient; the old version would sometimes take more than an hour to complete a run (and is run every hour). This was causing some people to miss summaries, especially those with America/Chicago timezones.
• INTERNAL: Fix issue with the GoAMZ MultiDel() call not working on S3 to remove files and photos when deleting a group.
• NEW: Hashtag list view shows mute/follow duration. Also, redid the mute/follow hashtag screens.
• CHANGE: For new donation requests, we have changed the fee structure: the Groups.io fee is 2%, in addition to Stripe's credit card fees.
• CHANGE: Display the default 'Elephant' group icon on the mobile groups page for groups without an icon set (replacing the group/pie chart icons).
• SYSADMIN: Upgraded the TLS certificates for the *.groups.io domains to use Let's Encrypt certs instead of the older Gandi issued cert, to fix the Safari cert warning.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Work towards supporting notifications.
• BUGFIX: The timezone listed when viewing an event or in the calendar popup sometimes did not match the text when creating the event (the UTC offset was the same, just the city names could be different, but correct).
• SYSADMIN: Prep work for adding a second email server.
• INTERNAL: Sped up deletion of photos and files when deleting a group.
• CHANGE: Formatting changes in the subgroup listing page.
• NEW: You can now toggle between the existing gallery view and a new list view in the Hashtags page.
• INTERNAL: Re-wrote several web templates to address some technical debt issues.

Take care everyone.

Mark

Changes to the site this week:

• CHANGE: Updates to the Members manual and Owners manual from Nina.
• SYSADMIN: Upgraded the kernel of mail01, to hopefully address slowness issue.
• CHANGE: When a group is set to Reply To Sender and also has Remove Other Reply Options set, do not add a List-Post header to individual messages.
• BUGFIX: Daily summaries included topics that had been muted.
• NEW: Remember the current calendar view.
• NEW: Revamped mobile Groups page.
• CHANGE: Made the subject lines of the subscribe/unsubscribe emails more useful.
• API: Added /getenterpriseinfo endpoint.
• BUGFIX: The subject wasn't logged in the activity log when deleting or rejecting a pending message.
• CHANGE: Relaxed the XML parsing strictness of RSS feeds to allow feeds that would otherwise be considered broken.

Take care everyone.

Mark

Changes to the site this week:

• BUGFIX: We were linking to incorrect messages in the activity log when someone was removed for marking a non-message as spam.
• CHANGE: Updates to the Members Manual.
• CHANGE: Unbolded all labels on the website and unbolded all panel titles.
• CHANGE: Various wording tweaks on the website to make things more consistent.
• BUGFIX: Fix Zoom meeting integration behavior when the Oauth token expires.
• BUGFIX: Generate a longer password when setting up a Zoom meeting.
• BUGFIX: Fix redirect when asked to log in to a group that has a subdomain.

Take care everyone.

Mark

Changes to the site this week:

• CHANGE: Allow background and mso-highlight CSS styles in messages.
• CHANGE: Various icon changes for more consistency.
• CHANGE: Updated the Members Manual with changes on Wiki page URLs.
• NEW: When a person is banned from Groups.io, we now log that in all groups they were a member of, and we remove all subscription data instead of just hiding it, which should keep sub counts correct.
• BUGFIX: The sending of a new confirmation email by a moderator using the 'Send Confirmation Email' button was not logged properly in the email delivery section.
• NEW: Logging to the activity log when we send the group guidelines out and when we send monthly reminder messages out.
• INTERNAL: Optimized a database call when generating digests.
• BUGFIX: Don't include links to profiles of banned members when viewing messages.
• CHANGE: When visiting a manual in the help center, display the table of contents.
• CHANGE: Wiki pages now use the wiki page ID instead of the title (with exceptions for _Sidebar/_Footer) to prevent link breakage when titles change.
• INTERNAL: A lot of work to enable distributed tracing of the internal systems.

Take care everyone.

Mark

Changes to the site this week:

5/1/20:

• NEW: Updated Help Center manuals with Nina's updates, including reformatted PDFs.
• SYSADMIN: Continued debugging issues with the new database server.

4/30/20:

• CHANGE: Documented photo/file upload size limits and increased them for premium/enterprise groups.
• SYSADMIN: Upgraded the web servers with more memory to deal with increased file/photo upload sizes.
• CHANGE: Rolled back the List-Id change from 4/25/20.

4/28/20:

• BUGFIX: Sorting by file type in the Files section would sometimes return the same results over multiple pages because our sort criteria wasn't stable.
• INTERNAL: We now clean up unused RSS feeds in the database.
• INTERNAL: On an edited message, check if the digest message exists already before adding it, to prevent postgres log messages.
• INTERNAL: Changed nginx log format back to combined to goaccess can parse user agents.
• INTERNAL: Work to make it easier to add additional FBL providers.

4/27/20:

• API: /getevent wasn't obeying the group calendar permissions setting.
• NEW: We now parse more information about which message was flagged as spam from an FBL report, and send that out to the user and group owners.

4/25/20:

• CHANGE: The List-Id email header now includes the group id, to better help us parse FBL reports.

Take care everyone.

Mark

This is a different #changelog from normal. Most everything outlined below represents internal changes to the site. This likely won't make sense to most people, but I'm writing this all down so that I can remember it in the future. The one new thing that will affect people are the new FBL subscriptions, outlined below.

I switched main database machines late last week because of a (minor) corruption issue. The new main database machine was running on the same type of hardware as the old machine, and I had tested it for several weeks beforehand. However, starting at the beginning of this week (we're busiest on Mondays), it was clear that for some reason the new database machine was not up to handling the load of the site. The site was crashing regularly, and I was getting paged in the middle of the night. It took me a few days to rule out other issues and come to the conclusion that it was a kernel issue. The new database machine was running an older Linux kernel, which normally shouldn't affect performance, but I believe in this case was. As part of my debugging process, I optimized many database access patterns, which will be great going forward. I have changed the Linux kernel on all database machines, and, combined with the other optimizations I did, I believe that the performance issues have been addressed. Hopefully.

At the same time, email to Comcast, Cox and GoDaddy was being delayed. I was finally able to establish contact with both Comcast and Cox and got the throttling lifted. As part of that, Comcast asked that I subscribe to their FBL (Feed Back Loop) service. An FBL is how we get notified when someone marks an email of ours as spam. So we are now receiving spam reports from several more email providers, including Comcast, Cox, Fastmail, Mail.ru, and others. The other recommendation that Comcast had was that we reuse SMTP connections when sending email to them, something we were not doing. So I spent the time to implement that. As of now, email to Comcast and Cox is flowing again. Email to GoDaddy/Secureserver is still being throttled and I have not had any luck contacting them as of yet.

Notes: karld refers to our email sending server, named after Karl 'The Mailman' Malone. forcesummaries is the program that generates summary emails. grafana is the package we use to generate internal monitoring graphs.

4/24/20:

• CHANGE: We now automatically delete drafts that are older than two weeks.
• INTERNAL: We now reuse SMTP connections when sending email to major providers, instead of opening a new connection for each email.
• INTERNAL: On advice from Comcast, subscribed to the Comcast FBL, along with several other FBLs.
• SYSADMIN: Upgraded the kernel in the main database machine to a more recent version.

4/23/20:

• INTERNAL: Added grafana annotations to all cronjobs.
• SYSADMIN: Reduced comcast.net karld concurrency from 10 to 5.
• INTERNAL: Improved our email monitoring dashboards to better track when providers are not accepting email from us.

4/22/20:

• INTERNAL: Optimized forcesummaries some more (and again fixed its use of replicas).
• INTERNAL: Use db replicas for wiki queries.
• INTERNAL: Use db replicas for some auth queries.
• SYSADMIN: Upgraded the kernel in db replica machines to a more recent version.
• SYSADMIN: Changed when we run the stats process on db05.
• SYSADMIN: Added pg_stats_activity counts for db04/db05.
• SYSADMIN: Reduced comcast.net karld concurrency from 20 to 10.
• SYSADMIN: Reduced cox.net karld concurrency from 10 to 5.

4/21/20:

• INTERNAL: Optimized the replacement of email addresses in messages with links to user profiles.
• INTERNAL: Fixed some database indices.

4/20/20:

• SYSADMIN: Adjusted main database parameters to deal with the current load.
• SYSADMIN: Moved pgbouncer off main database machine to its own dedicated instance.
• INTERNAL: Changed the forcesummaries process to use database replicas.
• INTERNAL: Added additional stats graphing to the internal monitoring dashboard.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Changed how we were limiting email sent to some domains, to use the MX server domain as a key for limiting instead of the receipient's domain. This will affect all messages to GoDaddy hosted domains, orange.fr/wanadoo.fr, earthlink/mindspring/netcom and a couple others. This will hopefully allow us to keep under per-connection limits imposed by these email service providers and avoid getting temporarily blocked for trying to send too much email at one time.
• INTERNAL: Technical debt cleanup of web page templates. An ongoing process.
• CHANGE: Updated the Help Center manuals with updates from Nina.
• API: /downloadarchives was not returning the messages in ascending date order.
• CHANGE: Allow 'list-style-type' styling in user generated content.
• CHANGE: Some formatting tweaks to the Member page, including displaying the member's timezone.
• CHANGE: Changed 'Unsubscribe' button in the Subscription page to 'Leave Group' to be more clear.
• SYSADMIN: Switched primary database machines because of continuing (minor) corruption issues.
• CHANGE: When viewing a member's information on mobile, changed the nav buttons at the top to a dropdown instead. Also fixed several navigation bugs when viewing past members.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Work to investigate which technologies to use for the app.
• BUGFIX: Misc improvements to formatting of the mobile website.
• NEW: (Re)launched the Zoom calendar integration.
• BUGFIX: The left chevron icon in the photo view was not vertically aligned correctly.
• BUGFIX: Paging backwards in the Emailed Photos photo viewer stopped working if you paged back at the first photo.
• API: Fixed pagination bugs with /getevents endpoint.
• CHANGE: Switchover to the new Help Center, with finalized Owners and Members manuals and FAQ.
• CHANGE: Adjusted the size and alignment of the lines separating a group's footer from the rest of the message footer information.
• BUGFIX: When creating a new plain text wiki page, the page would internally be marked as being HTML, so on future edits the HTML editor would be used.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Improvements to our log file grepping system to help track down problems.
• NEW: Many updates to the new Help Center, Owners Manual and Members Manual.
• CHANGE: Lots of little wording tweaks on the website to make things more consistent.
• CHANGE: For messages sent to the +owner address, delete any existing To header and replace it with listname+owner@groups.io, to make it more clear where the message was sent.
• API: Added /newgroupalias and /deletegroupalias endpoints.
• NEW: Added a search bar to the new group home page/did some rearraigning of the page (as seen on beta@).
• API: Added /unbanmember endpoint.
• BUGFIX: When uploading a directory to the Files section, the description was not added to the top level folder.
• BUGFIX: Repost was not working in the Topic view.
• NEW: Added Zoom integration to the calendar. You can now set up Zoom meetings when you create new events.
• CHANGE: Cap message subject line length at 1000 characters.
• CHANGE: For groups set to Reply To Sender, no longer add the CC header to point back to the group. When replying-to-all, it would create a duplicate mailing.
• CHANGE: Improved the formatting of the new Owners Manual PDF.
• SYSADMIN: Upgraded the machine instance responsible for sending email.
• INTERNAL: To display attachments on the website, now redirect to S3 in many cases instead of us fetching/returning the data.

Take care everyone.

Mark

Changes to the site this week:

• INTERNAL: Work on new Zoom integration with the calendar.
• CHANGE: Do not allow setting up of two factor authentication when you are logged in using Google or Facebook.
• API: Changing a subgroup's name using the /updategroup endpoint would fail.
• API: Removed bcc_me parameter from /postdraft as that was unused.
• BUGFIX: Private group guidelines were accessible to non-subscribers via the +guidelines email command.
• INTERNAL: More logging to track email delivery speed
• INTERNAL: Optimizations for lowering the time to deliver email
• BUGFIX: Removed do-nothing Edit Settings menu option when viewing moderators on the Members page.
• BUGFIX: ICS files containing yearly repeating events were not parsable.
• API: /newdraft endpoint now supports draft_type_post_to_sub draft_type.

Take care, everyone.

Mark

Changes to the site this week:

• INTERNAL: Work on new Help Center and code to convert the new manuals from Google Docs to HTML.
• CHANGE: In the database table page, replaced the individual visibility toggle buttons with a dropdown menu.
• NEW: Database address columns have a default country.
• CHANGE: Changed width of all selects on the website so that they are their natural width, not 100%, which was confusing to some and ugly to all.
• CHANGE: For Multiple Choice, Multiple Select columns, use a series of checkboxes instead of a HTML select element.
• CHANGE: Improved look of the database add row page.
• NEW: Database columns have a new description field.
• API: Added /getactivitylog endpoint.
• API: The docs incorrectly left off the /get prefix for the subgroupcategories endpoints.
• API: Passing in the type parameter to /getmembernotices would return a blank response.
• BUGFIX: The database map wasn't working for Enterprise domains.
• BUGFIX: If an address field in a database was set to a non-default color, the color would not show up in the add/edit row screen, potentially causing the labels to also disappear.

Stay safe everyone.

Mark

Changes to the site this week:

• INTERNAL: Work on a new help center.
• BUGFIX: If an event had '<<' in the name, the group event notice would not be generated correctly.
• BUGFIX: Export user wasn't generating ICS files correctly.
• API: Added email_address field to group object.
• BUGFIX: There was a bug with event RSVPs which could prevent some people from RSVPing to an event.
• BUGFIX: Removed duplicate Member Sync buttons from various settings pages. This had been renamed to Service Sync and placed elsewhere.
• API: Fixed sort for /getmembernotices endpoint.
• CHANGE: Separate the settings in the Default Sub Settings page into two panels for greater clarity.
• CHANGE: For moderator only hashtags, including Special hashtags, we now allow their use if they've been added to an approved pending message sent from a non-moderator. This means they were added by the approving moderator before the message was approved.

Have a good weekend everyone.

Mark