Topics

RQuantLib 0.4.7 and macOS

Dirk Eddelbuettel
 

Just a quick follow-up email saying Hi to the few folks who joined the list
over the last few days: welcome!

The RQuantLib 0.4.7 release did what it was supposed to do: restore the
full(er) functionality after some parts had been disabled when we had Windows
issues. Which, as we now know, were due to using an inappropriately built
Windows library to link against. So that is taken care of now -- again
thanks to Josh, Shrektan and everybody who chipped in with tests.

Which leaves macOS.

I do not have a mac and cannot drive this. But maybe we can coordinate it
here. Simon us the repository at https://github.com/s-u/recipes to provide
macOS library builds. We need to work out how to plug in there, maybe by
first testing library build steps on local machines (for those who have a
mac) and then PR it to him. That is essentially what happened on the Windows
side with Jeroen's rwinlib repo.

If you are willing and able to help, maybe particularly over these slower
end-of-year days, then please reply here. It would be a great help -- there
are always some new user for whom compilation of such a large library is a
challenge too many. If we can help them with CRAN binary packages, they
would clearly be better off.

Cheers, Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

I want to help. But I left my Mac at home and can only be able to test it until next Monday when I return.

Previously, I built RQuantLib on macOS by taking advantage of the static library of QL and Boost provided in Homebrew. 
Not sure https://github.com/s-u/recipes can use Homebrew or not .
If not, we may have to build QL from source and need more tests... 

Cheers, Xianying

Dirk Eddelbuettel
 

On 20 December 2018 at 18:26, shrektan wrote:
| I want to help. But I left my Mac at home and can only be able to test it until next Monday when I return.
|
| Previously, I built RQuantLib on macOS by taking advantage of the static library of QL and Boost provided in Homebrew. 
| Not sure https://github.com/s-u/recipes can use Homebrew or not .
| If not, we may have to build QL from source and need more tests... 

I think we need to look into building from source. Simon wants this 'from
source'.

The good news is that

i) these are mature libraries
ii) macOS has tested compilers
iii) from Boost we only need headers (unless we want to run unit tests for QL)

but I am unsure about

iv) quirks about creating binaries for macOS _with/for the R toolchain_

But we should be able to figure this out. CCing James aka @coatless who
knows quite a bit here and seems to have unlimited patience for macOS.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

David Winsemius
 

I will see if I can make sense of the Mac instructions this weekend. My experience with getting system dependencies to build from source has been mixed. When there are clear instructions about where _everything_ is supposed to be placed and what environment variables need to be in place _before_ the ./configure and make, make install steps are executed, then I have a good chance of success. If the authors just assume  that any idiot will know all these things, then I'm even less than that hypothetical idiot.  I now have a Linux version of Quantlib and Boost but only because of Dirk's binary package. The effort to install from source using the skimpy instructions on the Quantlib webpages seemed to be progressing  but apparently was put in the wrong directory or was missing needed configure arguments or environment variables (or something)  and ultimately fails the Bermudaswoption compile test.


Best;

David.

On 12/20/18 6:32 PM, Dirk Eddelbuettel wrote:
On 20 December 2018 at 18:26, shrektan wrote:
| I want to help. But I left my Mac at home and can only be able to test it until next Monday when I return.
|
| Previously, I built RQuantLib on macOS by taking advantage of the static library of QL and Boost provided in Homebrew.
| Not sure https://github.com/s-u/recipes can use Homebrew or not .
| If not, we may have to build QL from source and need more tests...

I think we need to look into building from source. Simon wants this 'from
source'.

The good news is that

i) these are mature libraries
ii) macOS has tested compilers
iii) from Boost we only need headers (unless we want to run unit tests for QL)

but I am unsure about

iv) quirks about creating binaries for macOS _with/for the R toolchain_

But we should be able to figure this out. CCing James aka @coatless who
knows quite a bit here and seems to have unlimited patience for macOS.

Dirk

Dirk Eddelbuettel
 

David and Xianying,

Much appreciate the offers for help.  I'd say just email here if/when you're stuck -- no point banging heads against the wall (says guy who banged head against a wall at work most of the day ;-) 

Dirk

shrektan
 

Hi Dirk and David,

I've done some tests and here's what I get for today.

1. The s-u/recipes currently need SIP(System Integration Protection) being disabled, which is not good for testing...
    I don't have an OSX VM so don't want to disable the SIP. 
    Luckily, my workaround https://github.com/s-u/recipes/issues/5 works for the QuantLib builds.

2. What's worse is that I don't think there's a way to download the boost header separately...
    It means we may have to bundle the boost header into the QuantLib's source tarball file and
    then upload to Github for every release in the future.
    Not sure if this is desirable.

3. Another way is to build the boost library (so QuantLib can 'depend' on it).
    The source code of the boost library doesn't contain the 'configure' file which s-u/recipes needs...
    I may have to intrude the source tarball and back to the previous situation again.

4. I can't really 'test' it because the environment on my machine may be different from s-u/recipes .


Cheers,
Xianying

Dirk Eddelbuettel
 

Hi Xianying,

On 24 December 2018 at 09:05, shrektan wrote:
| Hi Dirk and David,
|
| I've done some tests and here's what I get for today.
|
| 1. The s-u/recipes ( https://github.com/s-u/recipes ) currently need SIP(System Integration Protection) ( https://support.apple.com/en-us/HT204899 ) being disabled, which is not good for testing...
|     I don't have an OSX VM so don't want to disable the SIP. 
|     Luckily, my workaround https://github.com/s-u/recipes/issues/5 works for the QuantLib builds.

As a non-macOS user I can't help there.

| 2. What's worse is that I don't think there's a way to download the boost header separately...
|     It means we may have to bundle the boost header into the QuantLib's source tarball file and
|     then upload to Github for every release in the future.
|     Not sure if this is desirable.

That may be a problem. In the past we have bundled Boost for QuantLib to be
self-contained. We could try that again -- to keep it simple.

One alternative may be to rely on BH and point to its Boost headers. That
may lead to tinkering with the scripts as BH does not have any Boost
libraries and the default build _may_ involve boost-test.

| 3. Another way is to build the boost library (so QuantLib can 'depend' on it).
|     The source code of the boost library doesn't contain the 'configure' file which s-u/recipes ( https://github.com/s-u/recipes ) needs...
|     I may have to intrude the source tarball and back to the previous situation again.

Also not sure. Maybe we can have recipes involve other recipes?

Maybe ask Simon in another issue ticket?

| 4. I can't really 'test' it because the environment on my machine may be different from s-u/recipes ( https://github.com/s-u/recipes ) .

I don't follow. What is different? Is the a big concern or just a minor concern?

Thanks, Dirk

| Cheers,
| Xianying
|
|
|

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

Hi Dirk,

That may be a problem. In the past we have bundled Boost for QuantLib to be
| self-contained. We could try that again -- to keep it simple.

That would be great. I've just successfully built RQuantLib from source on my Mac,
based on the QuantLib library built by the receipts. The tests are all passed.

| One alternative may be to rely on BH and point to its Boost headers. That
| may lead to tinkering with the scripts as BH does not have any Boost
| libraries and the default build _may_ involve boost-test.

I'm confused. BH is an R package so it can only be used when building RQuantLib
from source, correct? Or you are saying we can use the boost header inside of the
source file of BH...

|
I don't follow. What is different? Is it a big concern or just a minor concern?

A minor concern, because as stated in QuantLib installation on Mac OS X ,
QL needs different environment variables for
a different version of OSX needs in order to compile.

In conclusion, if it's fine to use a bundled QL tarball, I can submit a PR to s-u/receipts,
which should be able to create a static library of QL on OSX... What's next?

Note, I notice a problem when building RQuantLib package from source on OSX, which is

clang: error: unsupported option '-fopenmp'

I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile. 

Regards,
Xianying



Dirk Eddelbuettel
 

Hi there,

On 24 December 2018 at 10:38, shrektan wrote:
| Hi Dirk,
|
| | That may be a problem. In the past we have bundled Boost for QuantLib to be
| | self-contained. We could try that again -- to keep it simple.
|
| That would be great. I've just successfully built RQuantLib from source on my Mac,
| based on the QuantLib library built by the receipts. The tests are all passed.

So that would be based on the recipe(s) you built?

| | One alternative may be to rely on BH and point to its Boost headers. That
| | may lead to tinkering with the scripts as BH does not have any Boost
| | libraries and the default build _may_ involve boost-test.
|
| I'm confused. BH is an R package so it can only be used when building RQuantLib
| from source, correct? Or you are saying we can use the boost header inside of the
| source file of BH...

We could point the QuantLib "recipe" (== Makefile etc) to the installed BH
package to use its copy of the headers. As opposed to requiring another full set.

| | I don't follow. What is different? Is it a big concern or just a minor concern?
|
| A minor concern, because as stated in QuantLib installation on Mac OS X ( https://www.quantlib.org/install/macosx.shtml ) ,
| QL needs different environment variables for a different version of OSX needs in order to compile.
|
| In conclusion, if it's fine to use a bundled QL tarball, I can submit a PR to s-u/receipts,
| which should be able to create a static library of QL on OSX... What's next?

That would be *huge*. We can then adjust RQuantLib as needed. In an ideal it
would just build.

| Note, I notice a problem when building RQuantLib package from source on OSX, which is
|
| clang: *error:* unsupported option '-fopenmp'
|
| I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
| So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile. 

Good. We can add a configure check to see if we are on an Architecture which
supports -fopenmp --- RcppArmadillo needed that too and we have something
working there. It is the same issue. Ie in RQuantLib I have

# See configure.in for how these variables are computed
PKG_CXXFLAGS=@CXXFLAGS@ -I../inst/include -I. -DBOOST_NO_AUTO_PTR $(SHLIB_OPENMP_CXXFLAGS)
PKG_LIBS=@LDFLAGS@ $(SHLIB_OPENMP_CXXFLAGS)

and SHLIB_OPENMP_CXXFLAGS is filled in by R ... when on macOS it should not :-/

But small fries. The bigger issue is that we seem to have recipes! Yay!

Dirk


--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

Hi Dirk, 

I have submitted the recipe PR in https://github.com/s-u/recipes/pull/7 . 

Cheers,
Xianying

Dirk Eddelbuettel
 

Hi Xianying,

On 24 December 2018 at 22:29, shrektan wrote:
| Hi Dirk, 
|
| I have submitted the recipe PR in https://github.com/s-u/recipes/pull/7 . 

That is great! We'll have to see what Simon says now.

I suspect the idea with recipes was to make it 'tamper-proof' by downloading
the tarball from the original location. That isn't exactly the same here so
Simon may hesitate. Let's see what he has to say.

Thanks so much for all the help so far. Great push in the right direction!

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

David Winsemius
 


On 12/24/18 10:38 AM, shrektan wrote:
Note, I notice a problem when building RQuantLib package from source on OSX, which is

clang: error: unsupported option '-fopenmp'

I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile. 
Regards, Xianying

Re: the lack of -fopenmp tag support in OSX. I got this error that when I tried to install on my MacPro that does have  quantlib installed (that box running El Capitan). According to what I read, that issue is supposedly solved by the use of more recent versions of the non-Apple clang. I'm not sure if it's just clang6 or perhaps either clang4 or clang6. You can get a clang4 from the libs directory

http://r.research.att.com/libs/   # I'm pretty sure that's where I got my boost and QuantLib packages

The newer clang is recommended for R 3.5.x:

http://r.research.att.com/libs/clang-4.0.0-darwin15.6-Release.tar.gz   # older version

clang6 is at the tools library: https://cran.r-project.org/bin/macosx/tools/  # for newer version

Dirk Eddelbuettel
 

On 25 December 2018 at 09:31, David Winsemius wrote:
|
| On 12/24/18 10:38 AM, shrektan wrote:
| > Note, I notice a problem when building RQuantLib package from source
| > on OSX, which is
| >
| > clang: *error: *unsupported option '-fopenmp'
| >
| > I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
| > So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile.
| > Regards, Xianying
|
| Re: the lack of -fopenmp tag support in OSX. I got this error that when
| I tried to install on my MacPro that does have  quantlib installed (that
| box running El Capitan). According to what I read, that issue is
| supposedly solved by the use of more recent versions of the non-Apple
| clang. I'm not sure if it's just clang6 or perhaps either clang4 or
| clang6. You can get a clang4 from the libs directory
|
| http://r.research.att.com/libs/   # I'm pretty sure that's where I got
| my boost and QuantLib packages
|
| The newer clang is recommended for R 3.5.x:
|
| http://r.research.att.com/libs/clang-4.0.0-darwin15.6-Release.tar.gz #
| older version
|
| clang6 is at the tools library:
| https://cran.r-project.org/bin/macosx/tools/  # for newer version

I am no authority here as I am not a macOS user. But I understand it's
complicated. See lines 67 to 106 in RcppArmadillo's configure.ac:
https://github.com/RcppCore/RcppArmadillo/blob/master/configure.ac#L67-L106
IIRC that was mostly @coatless aka James working this out.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

David Winsemius
 



On Dec 25, 2018, at 10:16 AM, Dirk Eddelbuettel <dirk.eddelbuettel@...> wrote:


On 25 December 2018 at 09:31, David Winsemius wrote:

| On 12/24/18 10:38 AM, shrektan wrote:
| > Note, I notice a problem when building RQuantLib package from source 
| > on OSX, which is
| >
| > clang: *error: *unsupported option '-fopenmp'
| >
| > I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
| > So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile.
| > Regards, Xianying 

| Re: the lack of -fopenmp tag support in OSX. I got this error that when 
| I tried to install on my MacPro that does have  quantlib installed (that 
| box running El Capitan). According to what I read, that issue is 
| supposedly solved by the use of more recent versions of the non-Apple 
| clang. I'm not sure if it's just clang6 or perhaps either clang4 or 
| clang6. You can get a clang4 from the libs directory

| http://r.research.att.com/libs/   # I'm pretty sure that's where I got 
| my boost and QuantLib packages

| The newer clang is recommended for R 3.5.x:

| http://r.research.att.com/libs/clang-4.0.0-darwin15.6-Release.tar.gz
| older version

| clang6 is at the tools library: 
| https://cran.r-project.org/bin/macosx/tools/  # for newer version

I am no authority here as I am not a macOS user.  But I understand it's
complicated. See lines 67 to 106 in RcppArmadillo's configure.ac:
https://github.com/RcppCore/RcppArmadillo/blob/master/configure.ac#L67-L106
IIRC that was mostly @coatless aka James working this out.

Dirk

Apologies if this is a duplicate. I found it in my drafts and do not see another copy of it in my Inbox.


My reading of that, and I can certainly be corrected, is that clang4 and above should have openmp support. 

My understanding of the error message is that sending -fopenmp to a non-supporting compiler is what is triggering the error.

I've installed the boost package v 1.47 from the att.research libs directory. That seems to meet the requirements of  QL


-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd@...




David Winsemius
 

On Dec 25, 2018, at 10:16 AM, Dirk Eddelbuettel <@eddelbuettel> wrote:


On 25 December 2018 at 09:31, David Winsemius wrote:
|
| On 12/24/18 10:38 AM, shrektan wrote:
| > Note, I notice a problem when building RQuantLib package from source
| > on OSX, which is
| >
| > clang: *error: *unsupported option '-fopenmp'
| >
| > I haven't figured out where SHLIB_OPENMP_CXXFLAGS gets defined.
| > So I manually removed SHLIB_OPENMP_CXXFLAGS in Makevars before compile.
| > Regards, Xianying
|
| Re: the lack of -fopenmp tag support in OSX. I got this error that when
| I tried to install on my MacPro that does have quantlib installed (that
| box running El Capitan). According to what I read, that issue is
| supposedly solved by the use of more recent versions of the non-Apple
| clang. I'm not sure if it's just clang6 or perhaps either clang4 or
| clang6. You can get a clang4 from the libs directory
|
| http://r.research.att.com/libs/ # I'm pretty sure that's where I got
| my boost and QuantLib packages
|
| The newer clang is recommended for R 3.5.x:
|
| http://r.research.att.com/libs/clang-4.0.0-darwin15.6-Release.tar.gz #
| older version
|
| clang6 is at the tools library:
| https://cran.r-project.org/bin/macosx/tools/ # for newer version

I am no authority here as I am not a macOS user. But I understand it's
complicated. See lines 67 to 106 in RcppArmadillo's configure.ac:
https://github.com/RcppCore/RcppArmadillo/blob/master/configure.ac#L67-L106
IIRC that was mostly @coatless aka James working this out.

Dirk
My reading of that code, and I can certainly be corrected, is that clang4 and above should have openmp support.

My understanding of the error message is that sending -fopenmp to a non-supporting compiler (such as is provided in the Xcode compilers) is what is triggering the error.

I've installed the boost package v 1.47 from the att.research libs directory. That seems to meet the requirements of QL.



--
David

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd


Dirk Eddelbuettel
 

Dear group,

I am truly sorry about how we continue to be _ignored_ by Simon on the PR
submitted by @shrektan on Dec 25: https://github.com/s-u/recipes/pull/3

I did not expect this, and would not have suggested doing work on this if I
had known. This is really sad. And I feel bad about this.

Again, sorry. I still hope it gets resolved "one day".

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

Hi Dirk,

Sorry for the late reply.  The email has been classified as a subscription by
my email service provider and placed into a directory which I rarely check...

No worries. It's a good opportunity for me to learn all of these stuff and I had fun.

In addition, I'm now quite familiar with compiling RQuantLib (or the QuantLib Library)
on Windows and OSX. At least I can provide the binaries to your drat repo whenever it's needed :D

Xianying Tan

Dirk Eddelbuettel
 

Howdy,

On 25 February 2019 at 01:57, shrektan wrote:
| Hi Dirk,
|
| Sorry for the late reply.  The email has been classified as a subscription by
| my email service provider and placed into a directory which I rarely check...

Ooops.

| No worries. It's a good opportunity for me to learn all of these stuff and I had fun.
|
| In addition, I'm now quite familiar with compiling RQuantLib (or the QuantLib Library)
| on Windows and OSX. At least I can provide the binaries to your drat repo whenever it's needed :D

Yes -- I was wondering about that too.

The new QuantLib 1.15 seems to work interchangeably with 1.14 but maybe we
should think about a drat repo given that Simon continue to ignore us and his
own recipes repo.

Dirk

|
| Xianying Tan
|
|
|

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd