Date   

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Tue, Dec 12, 2017 at 03:23 pm, Richard Russell wrote:
That will hopefully be rare, but it does mean that incorporating the 64-bit indicator in @platform% is not entirely free of unwanted side-effects.
Not so rare, as it turns out.  Having made that change, SDLIDE.bbc failed as a result of testing (in three places) the LS byte of @platform% for zero to mean Windows!  Trivial to fix of course, but indicative of the compatibility issues even this simple change might cause.

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Tue, Dec 12, 2017 at 01:41 pm, Mark Moulding wrote:
it won't affect any existing code that checks the platform, except on 64-bit versions where the programmer should be checking for other things anyway.
It doesn't follow that the programmer "should be checking for other things anyway".  Many existing programs are unaffected by the 32-bit vs 64-bit differences (if you've downloaded and tried bb64.exe - and I do hope you have - you should have discovered that the majority of example programs supplied with BBCSDL run perfectly well).  But they could still want to do something different when running on Windows (perhaps because of the different directory delimiter, or different C runtime) so might be testing the LS byte of @platform% to see if it is zero.  Such a program would conclude that the OS is not Windows when in fact it is.  That will hopefully be rare, but it does mean that incorporating the 64-bit indicator in @platform% is not entirely free of unwanted side-effects.

this means smallish stand-alone utilities that I want to be able to email and have run without requiring any installation
This is never going to be quite as straightforward with BBCSDL programs (whether 32 or 64-bits) because of the necessity to ship the SDL and related DLLs with your application - or at least require the end-user to download them himself.  However there are compensatory benefits such as cross-platform compatibility, access to a high-performance games engine without needing to write assembler code etc.

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Tue, Dec 12, 2017 at 10:54 am, Ric wrote:
I think that the method is not important, as long as it is consistent and if adding it to @platform% is the easiest to implement, then why not, there's no point in making life difficult for yourself.
I'm not keen on making design decisions based on expediency.  It's so easy to do something that you later regret, but cannot change because it would break existing programs (and there are several aspects of BBC BASIC - in both the original and my additions - that fall into that category); oh for a time machine!

For example after posting my last message I wondered whether it would be better to set bit 6 rather than bit 7 to indicate 64-bits (because it's equivalent to adding 64!).  So asking for a second opinion is, at least in part, of value in sharing the blame should the decision later be regretted.  ;-)

Richard.

Re: 64-bit BBC BASIC (again)

Mark Moulding
 

I'm an infrequent user of BB4W, and one who probably won't be affected by this change anyway - I use the Windows version exclusively, and don't foresee any time soon when Microsoft will drop 32-bit support.  That said, this approach (setting bit d7 of the @platform% value) seems like a sensible way to go - it won't affect any existing code that checks the platform, except on 64-bit versions where the programmer should be checking for other things anyway.

Incidentally, although I don't use BB4W too much, when I do it's because it's the perfect tool for the job.  In my case, this means smallish stand-alone utilities that I want to be able to email and have run without requiring any installation (and certainly without the enormous .Net quagmire).  Thanks so much for having created and maintained this product for so long - I truly do appreciate it.  (I suspect a lot of other lurkers like myself feel the same way.)
~~
Mark Moulding

Re: 64-bit BBC BASIC (again)

Ric
 

Richard,

I think that the method is not important, as long as it is consistent and if adding it to @platform% is the easiest to implement, then why not, there's no point in making life difficult for yourself.

Ric


----- Reply message -----
From: "Richard Russell" <news@...>
To: <bb4w@groups.io>
Subject: [bb4w] 64-bit BBC BASIC (again)
Date: Tue, Dec 12, 2017 16:21

On Fri, Dec 8, 2017 at 04:03 pm, Richard Russell wrote:
So this immediately raises one question: how should the '64-bitness' be made known to the program so that code can be made conditional in this way?
Does nobody have an opinion on this?  It's annoying that SDLIDE.bbc has a tendency to crash when run in 64-bits mode, and I know for sure that one reason is a structure that needs to be modified.  I'd therefore like to add some conditional code to that program to declare the appropriate structure for the current mode, but before I can do that I need to know how to make that determination in BASIC.

From an implementation point of view the easiest option would probably be to add the information to the least significant byte of @platform%.  Currently that can take only the values 0 (Windows), 1 (Linux), 2 (Mac OS) or 3 (Android) which requires only 2 bits out of the 8 available.  Perhaps the 32/64-bits state could be indicated in bit 7, so that 64-bit Windows would be &80, 64-bit Linux &81 etc.  How do people feel about that?

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Fri, Dec 8, 2017 at 04:03 pm, Richard Russell wrote:
So this immediately raises one question: how should the '64-bitness' be made known to the program so that code can be made conditional in this way?
Does nobody have an opinion on this?  It's annoying that SDLIDE.bbc has a tendency to crash when run in 64-bits mode, and I know for sure that one reason is a structure that needs to be modified.  I'd therefore like to add some conditional code to that program to declare the appropriate structure for the current mode, but before I can do that I need to know how to make that determination in BASIC.

From an implementation point of view the easiest option would probably be to add the information to the least significant byte of @platform%.  Currently that can take only the values 0 (Windows), 1 (Linux), 2 (Mac OS) or 3 (Android) which requires only 2 bits out of the 8 available.  Perhaps the 32/64-bits state could be indicated in bit 7, so that 64-bit Windows would be &80, 64-bit Linux &81 etc.  How do people feel about that?

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Sat, Dec 9, 2017 at 03:43 am, Jeremy Nicoll wrote:
unless one was exceptionally lucky (eg if the program
didn't use facilities provided through those variables).
I don't know about being "exceptionally lucky" but in this case, certainly, those members are unused by 'dropperz.bbc' and the change I listed is the only one required.  It's evident that some, at least, of the crashes in SDLIDE.bbc arise from a similar cause.

presumably how likely a crash is depends on what the storage area next to the area(s) allocated is being used for?
Of course that's true, but it's only going to determine how the program fails (e.g. whether it's a catastrophic crash or a trappable error) not whether it fails.  It's very unlikely indeed that a program will survive 8-bytes of heap being corrupted 'without damage'.

If someone is looking at a 32-bit program wondering if it will work in 64-bit mode, is there an easy way to check that?   Presumably in this case that AudioSpec structure corresponds to a defined data layout described somewhere? 
I stated in the original message that AudioSpec was an SDL2 structure so of course it is described in the online SDL 2.0 docs.  If you fancy writing a program (similar in some ways to the Windows Constants add-in utility) that scans a program for likely problematic structures, automatically looks them up in the appropriate online reference, and either reports where changes will be necessary or makes them automatically, I would be delighted.  Indeed that is exactly the sort of practical contribution that I am looking for from you and other members.

Does it matter much what the mechanism is? 

I think it does, yes. You can justifiably point to the existing mechanisms for detecting platform, CPU, OS etc. as being inconsistent and not being properly 'designed' from the start, but arbitrarily shoehorning the CPU 'bitness' into one of them chosen at random only makes things worse.  We have (in BB4W/BBCSDL) INKEY(-256), @platform%, *HELP and the assembler (e.g. assembling a 'nop' instruction and checking the code generated) as sources of information.  In addition there are API calls which can return relevant data.  So I think it's worth giving some thought to which of these would be most appropriate to convey the CPU bitness.

Of course until we have a working assembler the option of determining which kind of CPU it is by assembling some 'trial' instruction(s) isn't available, but since that is what I currently use to distinguish between x86 and ARM it might make sense to use it to determine the 'bitness' as well.

Richard.

Re: 64-bit BBC BASIC (again)

 

On Sat, 9 Dec 2017, at 00:03, Richard Russell wrote:
... so if the program needs to be compatible with both versions of BBC BASIC
it would have to be coded like this:

  IF bits% = 64 THEN
    DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%%, userdata%%}
  ELSE
    DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%, userdata%}
  ENDIF
and elsewhere in the program any logic that uses callback & userdata
would need
changed as well, unless one was exceptionally lucky (eg if the program
didn't use
facilities provided through those variables).

You said earlier that it wasn't surprising that BBC BASIC crashed after
the SDL wrote
an extra 8 bytes to the area(s) of storage concerned... but presumably
how likely a
crash is depends on what the storage area next to the area(s) allocated
is being used
for?

Some programs might just be 'unpredictable' but not crash. Some might
appear to
work even though some memory was being overwritten...


If someone is looking at a 32-bit program wondering if it will work in
64-bit mode,
is there an easy way to check that? Presumably in this case that
AudioSpec structure
corresponds to a defined data layout described somewhere? MSDN? Do
MS
list anywhere API calls whose data layouts change?


So this immediately raises one question: how should the '64-bitness' be
made known to the program so that code can be made conditional in this
way?  Should, for example, INKEY(-256) return a different code for
32-bits and 64-bits?   I would welcome suggestions.
Does it matter much what the mechanism is? The information needs to be
available somehow, and whatever you implement, people will program
around that.

But, I see from: http://beebwiki.mdfs.net/INKEY#Check_machine_type
that there's a long list of "machine types" many of which are not
machines at
all, but eg OS levels. OSBYTE 0 also seems to combine various types of
value.

I think it's a pity that that happened, although understandable. No one
knew
back in the mists of time that so many different systems, OSes etc would
be in
use so much later on. But limiting eg INKEY(-256) to returning a single
value
means it's messy if you have to start returning overlapping values, or
you have
to enumerate every combination of bitness, cpu type etc in a set of
values.

Maybe it would be better to have a separate BITNESS indicator?

--
Jeremy Nicoll - my opinions are my own.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Fri, Dec 8, 2017 at 07:59 pm, Cecil Bayona wrote:
It's impossible to create the assembler if one has no information on the internals of BBC Basic.
I really can't see why; the assembler is substantially independent of the rest of the interpreter (apart from needing to discover the values of variables, of course)
Obviously, aspects of the assembler which are independent of the CPU type - such as the formatting and outputting of the listing, creation of labels, differences between the different 'passes' depending on the value of OPT etc. - are not required since the existing code (either x86 or ARM) will be entirely suitable.  All I need is the instruction encoding part, so for example if one was writing the assembler in BASIC what I would be looking for is a single function such as the following:

      nbytes% = FNassemble(pointer%, instruction$)

which would encode the instruction contained in the string parameter instruction$ to a series of bytes stored in memory at the address given by the parameter pointer%, and return the number of bytes written to the caller.  So for example the following would write a single byte &90 and return the value 1:

      nbytes% = FNassemble(pointer%, "nop")

This requires no knowledge of the internals of BBC BASIC, only the ability to discover the values of variables and expressions (which you can do using EVAL) when they are included in the instruction string as operands:

      nbytes% = FNassemble(pointer%, "mov rax,myVariable")

I hope you didn't think that I was expecting a volunteer to reproduce those aspects of the assembler which already exist and work perfectly well.  It's only things which are unique to the CPU type, in this case the x86_64, that are needed, specifically the encoding of an instruction expressed in mnemonic form to its binary equivalent.

Richard.

Re: 64-bit BBC BASIC (again)

Cecil Bayona
 

That is not the information I wanted. What resources from BBC Basic will be available to someone that want's to tackle the task for example source code to the current assembler, information on how to hook up the new assembler to the 64 bit Basic?

It's impossible to create the assembler if one has no information on the internals of BBC Basic.

On 12/8/2017 4:16 PM, Richard Russell wrote:
On Fri, Dec 8, 2017 at 09:39 am, Cecil Bayona wrote:
I keep wondering if one decided to help with the assembler what
resources if any would be available to get one started.
There are any number of online instruction encoding references.  Here's one that Google found <http://wiki.osdev.org/X86-64_Instruction_Encoding>.
It was also suggested in another thread that it might be possible to adapt an existing open-source assembler, if its licence is sufficiently permissive.
Richard.
--
Cecil - k5nwa http://thepartsplace.k5nwa.com/

Re: 64-bit BBC BASIC (again)

Richard Russell
 

Here's a classic example of an unavoidable incompatibility.  I noticed that 'dropperz.bbc' crashes when run in 64-bit mode, and on investigation the culprit is this statement:

  DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%, userdata%}

This declares a structure used by SDL, involved with audio output (specifically, playing a WAV file).  But two members of the structure (callback and userdata)  are 32-bits in Win32 but 64-bits in x86_64!  So when SDL attempts to write to this structure it actually writes 8 bytes more than expected, and not surprisingly BBC BASIC crashes.  The solution is of course straightforward: change the declaration as follows:

  DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%%, userdata%%}

and indeed that stops 'dropperz' crashing.  But there's no way that this change can be made 'automatically', so if the program needs to be compatible with both versions of BBC BASIC it would have to be coded like this:

  IF bits% = 64 THEN
    DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%%, userdata%%}
  ELSE

    DIM AudioSpec{freq%, format%, channels&, silence&, samples{l&,h&}, size%, callback%, userdata%}
  ENDIF

So this immediately raises one question: how should the '64-bitness' be made known to the program so that code can be made conditional in this way?  Should, for example, INKEY(-256) return a different code for 32-bits and 64-bits?   I would welcome suggestions.

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Fri, Dec 8, 2017 at 09:39 am, Cecil Bayona wrote:
I keep wondering if one decided to help with the assembler what resources if any would be available to get one started.
There are any number of online instruction encoding references.  Here's one that Google found.

It was also suggested in another thread that it might be possible to adapt an existing open-source assembler, if its licence is sufficiently permissive.

Richard.

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Fri, Dec 8, 2017 at 09:39 am, Cecil Bayona wrote:
Internally what is the assembler written in, Basic, Assembler, or something else?
The 32-bit x86 assembler in BB4W is itself written in assembly language; the ARM assembler in the Android and Raspberry Pi editions of BBCSDL is written in C.  But I'm in no position to dictate the language in which an x86_64 assembler should be written, it could no doubt be translated if necessary.  In any case, assemblers are often dominated by data tables (for example a list of valid opcode and operand combinations) which are independent of language.  For example the BB4W assembler consists of approximately 3000 lines in data tables but only 1000 lines or so of code.

Richard.

Re: 64-bit BBC BASIC (again)

Cecil Bayona
 

Internally what is the assembler written in, Basic, Assembler, or something else?

Unfortunately I'm a newbie at BBC Basic, I wish I could help, it seems like a worthwhile project. Now that the basic software running as a 64 bit application a next helpful step to help would be a 64 bit assembler to allow interfacing to the OS.

I keep wondering if one decided to help with the assembler what resources if any would be available to get one started.

On 12/8/2017 6:31 AM, Richard Russell wrote:
On Thu, Dec 7, 2017 at 03:36 pm, Norman Vingoe wrote:
Once again Richard you have pulled a rabbit out of the hat.
I wouldn't go that far - after all I haven't actually *solved* any of the compatibility issues surrounding a 64-bit version.  However what I hopefully *have* done is to demonstrate that a 64-bit version of BBC BASIC should indeed be possible if these issues can be satisfactorily addressed.  In so doing I hope more effectively to engage people in actively thinking about them.
There was probably a greater response to the earlier thread than to any other subject discussed here in recent times, so let's see if we can direct that interest towards providing practical suggestions.  On the other hand if there's little or no take up - and it wouldn't be the first time - I won't pursue a 64-bit version any further.  The ball is in your court.
Richard.
--
Cecil - k5nwa http://thepartsplace.k5nwa.com/

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Fri, Dec 8, 2017 at 06:37 am, Kendall Castor-Perry wrote:

Didn't try the IDE and I'm not used to working in command line mode so wasn't able to probe deeply.
It's rare (but refreshing!) to find somebody whose earliest experiences of BBC BASIC were anything other than 'immediate mode', because that's all you got on the BBC Micro.  But feel free to use SDLIDE.bbc in 64-bits mode; as I said it will run although it's rather flaky.  Based on a quick experiment BBCEdit.bbc seems to run too, but I've not done any extensive testing.

However if you actually want to run BASIC programs from the IDE you'll have to rename (or copy) bb64.exe to bbcsdl.exe otherwise it won't know where to find the run-time engine. ;-)

just an inconsistency or intentional?
Intentional (you should find that the 32-bit version does exactly the same).  Within BBCSDL the delimiter is consistently a "/" because that's essential to ensure cross-platform compatibility (Windows is happy with "/" but Linux etc. are not happy with "\").  However Windows returns the 'current directory' string with "\" delimiters, as you would expect.  The result is the mixture you noticed.

Windows is the odd-one-out of the platforms supported by BBCSDL because unlike all the others it's not basically Linux under the hood.  The directory delimiter is just one of several features that work slightly differently in Windows (*dir listing the files in alphabetical order is another).

Richard.

Re: 64-bit BBC BASIC (again)

 

Downloaded and installed bb64, ran some basic programs (no assembler or SYS calls), worked as expected (also failed as expected when I didn't set things up properly).  Didn't try the IDE and I'm not used to working in command line mode so wasn't able to probe deeply.  When you do a *dir the resulting path contains \ as directory separator except for the very end where there's a /, just an inconsistency or intentional? -- K

Re: 64-bit BBC BASIC (again)

Richard Russell
 

On Thu, Dec 7, 2017 at 03:36 pm, Norman Vingoe wrote:
Once again Richard you have pulled a rabbit out of the hat.
I wouldn't go that far - after all I haven't actually solved any of the compatibility issues surrounding a 64-bit version.  However what I hopefully have done is to demonstrate that a 64-bit version of BBC BASIC should indeed be possible if these issues can be satisfactorily addressed.  In so doing I hope more effectively to engage people in actively thinking about them.

There was probably a greater response to the earlier thread than to any other subject discussed here in recent times, so let's see if we can direct that interest towards providing practical suggestions.  On the other hand if there's little or no take up - and it wouldn't be the first time - I won't pursue a 64-bit version any further.  The ball is in your court.

Richard.

Re: 64-bit BBC BASIC (again)

Norman Vingoe
 

Once again Richard you have pulled a rabbit out of the hat. I have had access to BBC Basic in various forms since my school first received a Model A, through Electron, Einstein, Z88 and 80386 PC to Android mobile and Hudl2.
However I'm currently limited until I've fixed a Windows 10 boot issue on my >1GB laptop, to Windows 32b on mY <1GB desktop PC.
So any 64b participation from myself will be delayed.

64-bit BBC BASIC (again)

Richard Russell
 

I make no apologies for raising this subject again, as it seems increasingly likely that it will have to be addressed somehow.  Apple's iOS is already 64-bits only, and they have announced that the next major release of Mac OS will no longer support 32-bit apps.  Some major Linux distributions have discontinued their 32-bit versions, and support for 32-bit apps on their 64-bit editions is reliant on installing 'multiarch', which can be tricky and which many users are reluctant to do.

At the moment it seems improbable that Android will drop support for 32-bit apps any time soon: it has always been a 'multi-architecture' OS with both x86 and ARM (and at one time even MIPS) binaries being supported.  However since Marshmallow it has introduced a rule that shared libraries must be Position Independent, and it's difficult and inefficient to create position-independent 32-bit x86 code (there are no program-counter-relative addressing modes).  Although compilers can often be configured to create such code, adapting assembly language code to be position-independent is all but impossible, and of course my x86 BBC BASIC interpreter is 100% assembler!

It's only Windows that is likely to continue to fully support 32-bit applications - and hence 'my' BBC BASIC - for the foreseeable future.  Several major Windows applications are still 32-bits, including Microsoft's own Visual Studio and parts of Internet Explorer.  But this is little consolation when all the recent BBC BASIC developments have been of the cross-platform editions (Windows, Linux, Mac OS, Android, Raspberry Pi).

In the earlier thread we discussed at length some of the problems related to creating a 64-bit version of BBC BASIC, not least that it is impossible to maintain full compatibility with the existing versions.  We considered some alternative approaches and their pros and cons.  But no clear front-runner emerged and no good solutions were forthcoming for some of the more intractable issues, such as the SYS statement.

So to help move this forward I have created a highly experimental 64-bit version of BBCSDL (for Windows only) which should have enough functionality to allow some practical testing to be carried out and some experience gained of the implications of a transition to 64-bits.  In this version I have made no attempt to tackle the major compatibility issues, for example @hwnd% and @memhdc% are still 32-bits whereas the handles returned from SDL may not be.  The SYS statement has limited functionality with no support at all for passing floats.  And of course there's no assembler!

This version can be downloaded as bb64.zip, its contents should be extracted to a directory created specifically for the purpose.  Do not put the files in the same folder as an existing (32-bit) BBCSDL installation since the 64-bit DLLs will overwrite the 32-bit ones and break everything.  bb64.exe is the executable and opens the same kind of 'immediate mode' window that BBCSDL does when not using one of the IDEs.  bb64 will run SDLIDE.bbc but it's rather flaky and prone to crashing.

Ideally I would like everybody to download this version and have a play.  Experiment with things that ought to work (let me know if they don't!) but more importantly think about those areas where full compatibility is impossible and make suggestions for how this can best be addressed.   Last, but not least, this version badly needs an assembler so it would be incredibly helpful if somebody could volunteer to write one.

Richard.

Re: Starting BB4W compiled program from Windows 10 task scheduler

J.G.Harston
 

Richard Russell wrote:
...
was this: "I figured it out. On the General Tab of the Task Scheduler
properties page, I had checked the "Run whether user is logged on or
not" box. Apparently, that starts the task in a separate session.
When I checked the "Run only when user is logged on" box instead, the
task does, indeed show up in the foreground."
Does that work for you?
Well, I can confirm it works for me, WinXP and Win7.

--
J.G.Harston - jgh@... - mdfs.net/jgh