Topics

mbox upload - dates are missing #mbox #ioimportpl

Andrea
 
Edited

Dear Lena,

we used your programm to read in the old mails delivered by the download
of the dashboard. The original dates are entered in message source (of
mbox) but are not delivered into the new groups mail archive but as
sent-date you get the upload date (of today, for instances). Now we have
the question if there is any possibility to enter the date of the
original mail into the uploaded old mails?

Many greetings and thank you in advance for any answer
Andrea


Am 30.11.19 um 21:18 schrieb Lena:

On Fri, Nov 29, 2019 at 11:49 PM, Simon Child wrote:

The blocker was that groups.io blocked message import from the script after 40
messages and told me to try again later. About an hour later I still couldn't
import any more messages from that email address
I added a delay 50 seconds between messages to avoid the block. The number 50 can be changed in the beginning of the program.

Duane
 

On Fri, Dec 6, 2019 at 01:43 PM, Andrea wrote:
The original dates are entered in message source (of
mbox) but are not delivered into the new groups mail archive but as
sent-date you get the upload date
Only a transfer can change the Sent Date.  Any upload/program will be based on the date/time sent.

Duane
--
Help: https://groups.io/static/help
GMF's Wiki: https://groups.io/g/GroupManagersForum/wiki
Search button at the top of Messages list
A few site FAQs: https://groups.io/static/pricing#frequently-asked-questions

Sugar Lopez
 

Hi

Sorry but I need to find out how to extract these type of files.

I downloaded  my messages from groups IO and when I tried to open the files I could not.

.mbox

How do open these files?

Sugar

 

Duane
 

On Fri, Dec 6, 2019 at 03:01 PM, Sugar Lopez wrote:

.mbox

How do open these files?

You need an mbox reader, such as Thunderbird.

Duane
--
Help: https://groups.io/static/help
GMF's Wiki: https://groups.io/g/GroupManagersForum/wiki
Search button at the top of Messages list
A few site FAQs: https://groups.io/static/pricing#frequently-asked-questions

Sugar Lopez
 

I don’t use thunderbird I use outlook, zip, 7zip or rar

So if I don’t have thunderbird, there is no other way to open and extract my messages?

 

That’s too bad.

 

Sugar

 

From: GroupManagersForum@groups.io <GroupManagersForum@groups.io> On Behalf Of Duane
Sent: Friday, December 6, 2019 1:20 PM
To: GroupManagersForum@groups.io
Subject: Re: [GMF] mbox upload - dates are missing

 

On Fri, Dec 6, 2019 at 03:01 PM, Sugar Lopez wrote:

.mbox

How do open these files?

You need an mbox reader, such as Thunderbird.

Duane
--
Help: https://groups.io/static/help
GMF's Wiki: https://groups.io/g/GroupManagersForum/wiki
Search button at the top of Messages list
A few site FAQs: https://groups.io/static/pricing#frequently-asked-questions

Bruce Bowman
 

On Fri, Dec 6, 2019 at 05:35 PM, Sugar Lopez wrote:
I don’t use thunderbird I use outlook, zip, 7zip or rar
Sugar -- A simple Google search turns up several utilities that will convert an mbox file to PST so it can be read by Outlook.

https://www.google.com/search?hl=en&q=mbox+converter+pst+outlook

Bruce

Sugar Lopez
 

Thank you so much.

I truly appreciate that

Sugar

From: GroupManagersForum@groups.io <GroupManagersForum@groups.io> On Behalf Of Bruce Bowman
Sent: Friday, December 6, 2019 2:42 PM
To: GroupManagersForum@groups.io
Subject: Re: [GMF] mbox upload - dates are missing

 

On Fri, Dec 6, 2019 at 05:35 PM, Sugar Lopez wrote:

I don’t use thunderbird I use outlook, zip, 7zip or rar

Sugar -- A simple Google search turns up several utilities that will convert an mbox file to PST so it can be read by Outlook.

https://www.google.com/search?hl=en&q=mbox+converter+pst+outlook

Bruce

 

On Fri, Dec 6, 2019 at 09:43 PM, Andrea wrote:

Dear Lena,

we used your programm to read in the old mails delivered by the download
of the dashboard. The original dates are entered in message source (of
mbox) but are not delivered into the new groups mail archive but as
sent-date you get the upload date (of today, for instances). Now we have
the question if there is any possibility to enter the date of the
original mail into the uploaded old mails?
My program transmits original Date, but groups.io ignores it. I added an option (turned on by default) to insert original Date into beginning of message body.

I also fixed a bug (a deleted message caused next messages in a mbox file to be skipped).

Lena

Ingrid
 

Dear Lena,

so many thanks to you for having done this (and in not more than some hours - at 8 in the evening I was talking on the phone with Andrea about this problem, when I went to my notebook in the morning changes were ready!)

It is wonderful! Now our members have same information as in original message! (We could not make transfer, because our owner had died and so your program is like heaven-sent for us).

Kind regards
Ingrid

 

Sandi D.
 

On Fri, Dec 6, 2019 at 04:01 PM, Sugar Lopez wrote:
How do open these files?
I use Postbox, on my Mac. It's a mail Software program also available for Windows. If you want a program to store them and be able to search for them and email them to others, consider Postbox. It might be overkill for your needs. 
Postbox mbox imports

I requested an archive download in a Gmail account I wanted to close. It came to me in mbox file. I opened Postbox and created a new Folder. Then went to File -->Import and selected the FOLDER containing all the mbox files. They were all imported into the new Postbox folder and positioned under their appropriate folders and subfolders.

Postbox has extensive search functions that work well in tracking down archived messages. It can search in specific folders or across all my accounts. I can open and forward messages to others. I can move individual files across IMAP and POP accounts and sync them online in those active accounts if I choose.

Postbox preserved all the images and attachments to each message and I can download the messages and their attachments to my computer in a readable format. 

--
Sandi Dickenson
Moderator of ASG Volunteers Group

Andrea
 

Dear Lena,

thank you so much for the great work you have done. Now it works
perfekctly with the date!

Many greetings
Andrea

Andrea
 

Dear Lena,

thanks so much for your work!

Most times your new implementation works, we worked with it in the
meantime. But for a reason we do not understand this does not always
work, but only in about 50% of the messages. No systematic reason is
understood by us.

And then - we have a problem with "bounced" Subjects, even if they are
"no mail"-subjects. In this cases following happens:
Programm log says: "after sending message xxx: 500 ... is bouncing..."
and then the whole programm stops.

Is there any workarround for us that you can advice? We thought about
deleting all bounced subjects, but this is not exactly what we would
like to do...

Many greetings and again thanks for your great job with this programm,

Andrea


Am 07.12.19 um 00:18 schrieb Lena:

On Fri, Dec 6, 2019 at 09:43 PM, Andrea wrote:

Dear Lena,

we used your programm to read in the old mails delivered by the download
of the dashboard. The original dates are entered in message source (of
mbox) but are not delivered into the new groups mail archive but as
sent-date you get the upload date (of today, for instances). Now we have
the question if there is any possibility to enter the date of the
original mail into the uploaded old mails?
My program transmits original Date, but groups.io ignores it. I added an option (turned on by default) to insert original Date into beginning of message body.

I also fixed a bug (a deleted message caused next messages in a mbox file to be skipped).

Lena


 

On Wed, Dec 11, 2019 at 10:30 PM, Andrea wrote:

Most times your new implementation works, we worked with it in the
meantime. But for a reason we do not understand this does not always
work, but only in about 50% of the messages.
What exactly doesn't work? I fixed a bug (a deleted message caused all next messages in a mbox file to be skipped).

And then - we have a problem with "bounced" Subjects, even if they are
"no mail"-subjects. In this cases following happens:
Programm log says: "after sending message xxx: 500 ... is bouncing..."
and then the whole programm stops.
Download the memberlist.csv file again from Members - Download.

--
Lena

Ingrid
 

On Thu, Dec 12, 2019 at 03:55 PM, Lena wrote:

Download the memberlist.csv file again from Members - Download.

Yes, this was functioning. Thank you!


On Wed, Dec 11, 2019 at 10:30 PM, Andrea wrote:

Most times your new implementation works, we worked with it in the
meantime. But for a reason we do not understand this does not always
work, but only in about 50% of the messages.
  Your answer: 
  "What exactly doesn't work? I fixed a bug (a deleted message caused all next messages in a mbox file to be skipped)."

It's about the feature you had implemented which is writing "From" and "Date" in the beginning of the message.
In about 40% to 35% of messages this functions. In about 60% to 65%, there is neither a "From" nor a "Date". In these more than half of cases without "Date" I can't see any pattern. The appearence of „Date“ or no „Date“ seems to be absolutly random.
Have you any idea to solve this?

And there is another big problem: 
Sometimes one or the other message is not being uploaded. I did not care to much about that. But today the program imported a message from 27
th December and the next imported one was from 24th August of the following year. So it skipped more than 2000 messages!
And again there is no hint why the program would have done this. 
(Luckily I had made a subgroup for testing the program!)

I hope, you can help us!

Ingrid

 

On Tue, Dec 17, 2019 at 07:07 PM, Ingrid wrote:

It's about the feature you had implemented which is writing "From" and "Date"
in the beginning of the message.
In about 40% to 35% of messages this functions. In about 60% to 65%, there is
neither a "From" nor a "Date".
it skipped more than 2000 messages!
I fixed these two bugs. Please download the program again.

(Luckily I had made a subgroup for testing the program!)
Already imported messages are skipped by groups.io if you try to import them again into the same group or subgroup. Perhaps you'll have to delete that test subgroup and create it again.

Also, delete the imported.txt file.

--
Lena

Mike Beaty
 

I too am using ioimport.pl and am having some success. I understand that there is a newly-released version 5, and I will download it and use it.

I have several questions:
1. I see the needchange.txt file. Of ~2500 subscribers, I have ~270 entries in this file. Do I need to resolve all issues in this file for the program to succeed? What happens if I don't?
2. I got an error when I tried to use the map.txt file. I tried with both a comma and a space.

I want to import 10K+ messages, and would prefer to only do this once. :-)

Thanks very much to Lena for creating and assisting with this program!

Mike

Bruce Bowman
 

On Fri, Dec 20, 2019 at 03:05 PM, Mike Beaty wrote:
I want to import 10K+ messages, and would prefer to only do this once.
Mike -- I suggest you create a subgroup and practice with that until you get things working the way you want.

Subgroups are easily deleted, thousands of messages are not.

Bruce

Mike Beaty
 

I had ~600 topics. I used an auto-mouse click program to delete them all :-).

Per your suggestion, I created a subgroup, but it had only the owner shared from the main group. I would have to re-add all 2500 members, and they would be notified of the subscription, unless there's a way around this or I'm misunderstanding something.

ioimport.pl now shows my four mbox files contain a total 10324 messages (deleted messages excluded)

1515 email addresses need to be changed, see needchange.txt
Of those, 735 are NotMember, 735 are NewUserModerated, 39 are Bouncing, 3 Bounced, and 3 Pending
Please tell me I don't need to resolve all of these to have a successful import!

Thanks,

Mike

Christopher Warrington
 

On 2019-12-20 at 11:47:24 PM, Mike Beaty <yahoogroups@...> wrote:

Per your suggestion, I created a subgroup, but it had only the owner
shared from the main group. I would have to re-add all 2500 members, and
they would be notified of the subscription, unless there's a way around
this or I'm misunderstanding something.
I read the suggestion, rather, to use the subgroup to test the import
process and work out any kinks (e.g., the needchange.txt question you have).
If something goes awry, you can delete the subgroup and start over: no need
to deal with deleting individual messages, even with automation. After
getting the import to work they way you want, you can import to your main
group.

--
Christopher W. <lists@...>

Ingrid
 

Program works perfectly now! 
Thank you so much for fixing the bugs and doing it so promptly!

Ingrid