Finally got this working, reporting my issues here incase someone with similar issues stumbles upon this thread.
The T54W phones would not download the xml files by https (may be because I do not have a valid certificate setup). I could via the xml files in browser on https but phones would wait awhile and fail.
I am doing auto provisioning via http, but my default admin port was set to 8080. I could not download xml files via http with admin port on 8080.
Once I changed the admin port back to default 80, I was able to download xml files via http.
Phones can now download the php/xml files via http.
I changed permissions for the php files back to rw/rw/r with owner asterisk:asterisk
Auto Provisioning was another headache. Eventually I found out that I had the config settings wrong, was using features.remote_phonebook.data.1.url instead of the correct remote_phonebook.data.1.url
My working basefile edits (using EPM):
directory_setting.remote_phone_book.enable = 1
remote_phonebook.data.1.name = Internal
remote_phonebook.data.1.url = http://<pbx-ip>/ylab.php
features.remote_phonebook.enable = 1
remote_phonebook.data.2.name = External
remote_phonebook.data.2.url = http://<pbx-ip>/cm_to_yl_ab.php
With directory_setting.remote_phone_book.enable = 1
You do not need the favorite_settings.xml to enable the remote directory. But it will put local directory as the first option and remote phonebook as the second option after hitting the directory key.
I have added this code to change the order:
directory_setting.local_directory.priority = 2
directory_setting.remote_phone_book.priority = 1
Or to get rid of the local directory
directory_setting.local_directory.enable = 0
Yealink has a Config Parameter Excel file that has all the config parameters and reading it is what lead me to find my mistake.