一.window安装scrapy
1.先安装twisted
下载链接:https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
找跟你系统对应版本
我的是Twisted-18.9.0-cp36-cp36m-win_amd64.whl下载下来
先安装pip3 install  +你twisted的路径
2.安装scrapy
pip3 install scrapy(直接安装可能会报错error: Microsoft Visual C++ 14.0 is required.解决方式看1)

报错解决:
如果出现:ModuleNotFoundError: No module named 'win32api'
则安装win32api, 下载链接:  http://sourceforge.net/projects/pywin32/files%2Fpywin32/
（下载适合你的Python版本）我是安装这个pywin32-221.win-amd64-py3.6.exe
安装pywin32出现报错--Python version 3.6 required, which was not found in the registry
解决:创建register.py,执行他 import sys
try:
    from winreg import *
except ImportError:
    from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\\Python\\Pythoncore\\{0}\\".format(version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "{0};{1}\\Lib\\;{2}\\DLLs\\".format(
    installpath, installpath, installpath)
def RegisterPy():
    try:
        reg = OpenKey(HKEY_CURRENT_USER, regpath)
    except EnvironmentError as e:
        try:
            reg = CreateKey(HKEY_CURRENT_USER, regpath)
            SetValue(reg, installkey, REG_SZ, installpath)
            SetValue(reg, pythonkey, REG_SZ, pythonpath)
            CloseKey(reg)
        except:
            print("*** Unable to register!")
            return
        print("--- Python", version, "is now registered!")
        return
    if (QueryValue(reg, installkey) == installpath and
        QueryValue(reg, pythonkey) == pythonpath):
        CloseKey(reg)
        print("=== Python", version, "is already registered!")
        return
    CloseKey(reg)
    print("*** Unable to register!")
    print("*** You probably have another Python installation!")
if __name__ == "__main__":
    RegisterPy()

二.ubuntu安装scrapy
1.创建虚拟环境
mkvirtualenv spider_py3 -p python3
注意:可能出现问题,虚拟环境无法创建,阻塞住了
解决:升级pip,出现Successfully installed pip-19.0.2成功安装再重新创建
sudo pip3 install --upgrade pip

2.在虚拟环境里安装scapy
(推荐1.6.0,可能由于版本原因导致爬取数据编码为unicode,且创建项目无中间件文件)
pip install scrapy==1.6.0

一.window安装scrapy
1.先安装twisted
下载链接:https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
找跟你系统对应版本
我的是Twisted-18.9.0-cp36-cp36m-win_amd64.whl下载下来
先安装pip3 install  +你twisted的路径
2.安装scrapy
pip3 install scrapy(直接安装可能会报错error: Microsoft Visual C++ 14.0 is required.解决方式看1)

报错解决:
如果出现:ModuleNotFoundError: No module named 'win32api'
则安装win32api, 下载链接:  http://sourceforge.net/projects/pywin32/files%2Fpywin32/
（下载适合你的Python版本）我是安装这个pywin32-221.win-amd64-py3.6.exe
安装pywin32出现报错--Python version 3.6 required, which was not found in the registry
解决:创建register.py,执行他 import sys
try:
    from winreg import *
except ImportError:
    from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\\Python\\Pythoncore\\{0}\\".format(version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "{0};{1}\\Lib\\;{2}\\DLLs\\".format(
    installpath, installpath, installpath)
def RegisterPy():
    try:
        reg = OpenKey(HKEY_CURRENT_USER, regpath)
    except EnvironmentError as e:
        try:
            reg = CreateKey(HKEY_CURRENT_USER, regpath)
            SetValue(reg, installkey, REG_SZ, installpath)
            SetValue(reg, pythonkey, REG_SZ, pythonpath)
            CloseKey(reg)
        except:
            print("*** Unable to register!")
            return
        print("--- Python", version, "is now registered!")
        return
    if (QueryValue(reg, installkey) == installpath and
        QueryValue(reg, pythonkey) == pythonpath):
        CloseKey(reg)
        print("=== Python", version, "is already registered!")
        return
    CloseKey(reg)
    print("*** Unable to register!")
    print("*** You probably have another Python installation!")
if __name__ == "__main__":
    RegisterPy()

二.ubuntu安装scrapy
1.创建虚拟环境
mkvirtualenv spider_py3 -p python3
注意:可能出现问题,虚拟环境无法创建,阻塞住了
解决:升级pip,出现Successfully installed pip-19.0.2成功安装再重新创建
sudo pip3 install --upgrade pip

2.在虚拟环境里安装scapy
(推荐1.6.0,可能由于版本原因导致爬取数据编码为unicode,且创建项目无中间件文件)
pip install scrapy==1.6.0